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ABSTRACT 


This  thesis  investigates  the  visual  search  process  and  the  effect  of  contextual 
information  on  the  search  process  in  an  urban  combat  environment.  High  resolution 
combat  simulation  models  implement  a  parallel  sweeping  or  “windshield  wiper”  search 
process  that  is  not  representative  of  human  search  behavior.  Furthermore,  combat  models 
do  not  account  for  additional  situational  awareness  in  the  form  of  contextual  information. 
This  thesis  proposes  a  Discrete  Myopic  Search  model,  which  provides  a  statistical  model 
based  on  human  performance  data.  This  model  prioritizes  search  effort  where 
participants  believe  that  targets  are  most  likely  to  occur.  Nineteen  volunteers  searched  16 
static  urban  scenes  with  zero  to  five  targets.  These  data  formed  the  probabilities  that  a 
target  is  located  in  each  cell  in  each  discretized  scene.  The  Discrete  Myopic  Search 
model  chooses  the  cell  with  the  highest  probability  for  each  discrete  look.  Hypothesis 
testing  on  experimental  data  revealed  a  nearly  20%  increase  in  search  performance  of  the 
Discrete  Myopic  Search  model  over  the  windshield  wiper  model.  Further  investigation 
revealed  a  significant  change  in  search  behavior  and  detection  performance  based  on  the 
addition  of  contextual  information.  This  research  shows  that  combat  models  should 
prioritize  search  patterns  and  account  for  added  situational  awareness. 
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EXECUTIVE  SUMMARY 


Combat  simulation  models  are  widely  used  for  both  training  and  developing 
courses  of  action.  These  reasons  show  the  importance  of  valid  output  of  such  models. 
Unfortunately,  these  models  do  not  implement  a  visual  search  process  that  is 
representative  of  a  human  soldier.  The  order  in  which  a  simulation  model  detects  targets 
is  not  consistent  with  the  order  in  which  a  human  detects  targets.  If  a  human  does  not 
engage  the  most  dangerous  target  first,  he  will  most  likely  become  a  casualty.  If  a 
brigade  or  division  of  soldiers  does  not  engage  targets  in  an  appropriate  manner,  the 
battle  may  be  lost.  Even  a  small  probability  of  error  can  quickly  change  the  outcome  of  a 
battle  when  the  number  of  soldiers  is  very  large.  This  thesis  shows  the  flaw  in  the  search 
pattern  used  by  current  simulation  models  and  proposes  a  Discrete  Myopic  Search  model. 
The  presented  model  is  validated  using  experiment  data  from  observation  of  human 
subjects  and  contrasted  with  the  currently  implemented  model  using  a  population 
proportion  hypothesis  test.  The  analysis  highlights  a  nearly  20%  improvement  in  the 
correct  prediction  of  the  target  detection  order  when  utilizing  the  Discrete  Myopic  Search 
model.  Such  an  improvement  will  undoubtedly  change  the  mind  of  decision  makers  for 
certain  courses  of  action. 

Further,  simulation  models  currently  do  not  account  for  additional  situational 
awareness  of  individual  entities  in  the  form  of  contextual  information.  This  thesis  shows 
that  such  information  not  only  changes  the  search  behavior  of  human  participants,  but 
also  affects  detection  performance.  Sharing  information  among  soldiers  is  crucial  in  an 
urban  combat  environment.  Simulation  models  should  also  reflect  this  role  of 
information  sharing  among  individual  searching  entities. 

Current  U.S.  Army  doctrine  prescribes  a  thorough  search  technique  for  operations 
in  rural  terrain.  The  doctrine  explains  that  urban  terrain  is  more  complex,  but  stops  short 
of  prescribing  a  search  technique.  This  thesis  uses  experiment  data  to  show  that  typical 
rural  search  techniques  are  not  sufficient  for  urban  terrain.  It  goes  on  to  recommend 
language  for  further  investigation  and  consideration  in  future  doctrinal  manuals  regarding 
search  in  an  urban  environment. 
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I. 


INTRODUCTION 


A.  MOTIVATION 

Imagine  a  young  soldier  providing  security  over  his  sector  in  the  streets  of  an 
urban  combat  zone.  Three  targets  present  themselves  simultaneously,  each  of  them 
clearly  an  enemy  combatant.  The  first  target  is  approximately  250  meters  to  the  soldier’s 
eleven  o’clock.  This  target  is  oriented  in  a  direction  which  poses  no  immediate  threat  to 
the  soldier.  The  second  target  is  approximately  200  meters  to  the  soldier’s  front  and 
clearly  has  not  yet  observed  the  soldier.  The  last  target  is  25  meters  to  the  soldier’s  two 
o’clock  and  is  raising  his  weapon  to  engage  the  soldier.  Unfortunately,  the  soldier  is 
programmed  to  implement  a  predetermined  search  pattern,  which  sweeps  from  left  to 
right.  Before  he  can  engage  the  target  which  is  the  highest  threat,  he  must  first  adjudicate 
the  two  very  low  threat  targets.  The  third  target  engages  and  kills  the  soldier  before  the 
soldier  ever  acquires  the  target. 

Now  imagine  the  same  soldier  in  a  similarly  hostile  urban  situation.  This  time 
there  is  one  target  200  meters  to  the  front  on  a  rooftop  preparing  the  engage  the  soldier. 
The  soldier  implements  a  search  pattern  in  which  he  considers  100-meter  strips  of  terrain 
from  left  to  right  and  right  to  left  from  near  to  far  overlapping  strips  on  each  pass.  The 
terrain  to  his  immediate  front  consists  of  a  road  and  buildings  on  either  side  with  closed 
doors  and  windows,  none  of  which  pose  any  immediate  threat.  Just  as  in  the  first  case, 
the  target  engages  and  kills  the  soldier  before  the  soldier  can  acquire  the  target. 

The  first  situation  describes  the  search  pattern  implemented  by  high  resolution 
combat  simulation  models.  The  second  situation  describes  a  search  pattern  prescribed  by 
several  U.S.  Army  field  manuals.  In  both  cases  the  soldier  loses  the  fight.  Urban  terrain 
contains  vast  amounts  of  clutter,  provides  exceedingly  more  occlusions  than  rural  terrain, 
and  requires  a  more  deliberate  search  technique.  This  thesis  proposes  an  urban  search 
technique  based  on  the  likelihood  of  a  target’s  presence  at  any  point  in  a  given  field  of 
view.  This  technique  prioritizes  the  order  of  a  searcher’s  gaze  pattern  based  on  the 
probability  that  a  target  is  present. 
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B.  BACKGROUND 


At  the  most  basic  level  of  modem  tactical  ground  combat,  an  individual 
combatant  seeks  to  destroy  his  enemy  in  close  battle.  Fundamental  to  a  combatant’s 
ability  to  destroy  his  enemy  is  the  search  and  target  acquisition  process.  A  combatant 
must  first  search  for  a  stimulus,  identify  any  discovered  stimulus  as  an  enemy  combatant, 
determine  whether  or  not  the  situation  satisfies  the  Rules  of  Engagement,  and  ultimately 
engage  the  target.  This  research  investigates  the  visual  search  patterns — the  cornerstone 
of  the  search  and  target  acquisition  process — of  an  individual,  dismounted  soldier  in  an 
urban  environment. 

This  study  uses  eye  tracking  data  to  investigate  how  an  individual  with  military 
training  searches  a  static  scene  to  identify  hostile  targets.  The  data  include  x  and  y  pixel 
coordinates  of  the  gaze  on  a  computer  screen  of  each  individual  participant  at  a  rate  of  60 
data  points  per  second.  In  addition,  mouse  click  data  indicate  whether  the  participant 
correctly  identifies  a  target  or  if  he  falsely  identifies  a  stimulus  that  is  not  a  target.  A 
miss  results  when  the  participant  fails  to  identify  a  target.  These  data  form  the  basis  for 
the  measures  of  effectiveness  with  which  this  thesis  analyzes  and  compares  current  and 
proposed  search  models. 

More  precisely,  this  research  focuses  on  the  fixation  data  derived  from  the  eye 
tracking  information  and  the  probabilities  of  detection  derived  from  the  mouse  click  data 
to  formulate  a  Myopic  Allocation  of  Search  Effort  model.  This  model  determines  the 
optimal  allocation  of  search  effort  for  each  of  a  finite  number  of  discrete  looks  at  a 
partitioned  image.  This  thesis  seeks  to  improve  upon  the  search  process,  that  is,  the  order 
of  target  detections,  rather  than  the  detection  statistics  themselves;  detection  probabilities 
derived  from  experimental  data  remain  fixed  throughout  the  model  comparison. 

1.  Search  and  Target  Acquisition 

Search  is  defined  herein  as  the  process  of  viewing  a  search  field  to  locate  a  target. 
Target  acquisition  is  all  processes  needed  to  locate  a  target  and  discriminate  it  to  a 
desired  level.  These  processes  include  detection,  classification,  recognition,  and 
identification  (Vaughan,  2006). 
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Detection  is  the  determination  that  an  object  exists  such  that  it  is  distinct  from  its 
surroundings.  Classification  is  a  decision  whether  or  not  the  object  belongs  to  a  set  of 
targets  or  non-targets.  Recognition  is  a  process  which  determines  the  specific  functional 
category  to  which  the  object  belongs.  Identification  is  the  most  detailed  level  of 
discrimination  and  leads  to  the  decision  of  whether  or  not  a  target  is  hostile  (Vaughan, 
2006). 

The  Search  and  Target  Acquisition  Process  is  essential  to  this  research  and  critical 
to  a  successful  combat  operation.  Timely  and  accurate  implementation  of  this  process  by 
an  individual  entity  in  a  combat  simulation  model  or  an  individual  soldier  in  the  streets  of 
a  hostile  urban  environment  can  prove  the  difference  between  mission  success  or  failure. 

2.  Search  in  Current  Combat  Models 

Current  high  resolution  combat  simulation  models  such  as  OneSAF  and 
COMBATXXI  implement  a  specific,  discrete  search  process.  This  process  begins  by 
dividing  a  field  of  regard  (FOR)  into  discrete,  adjacent,  and  non-overlapping  fields  of 
view  (Harrington,  2009).  The  models  then  apply  acquisition  methodology,  described 
below,  to  each  FOV  in  a  “windshield  wiper”  pattern  (TRAC  WSMR,  2008).  Figure  1 
displays  a  flow  chart  of  the  FOV  search.  Note  that  this  search  is  for  only  one  FOV  in  the 
FOR.  Once  this  search  is  complete,  the  model  moves  to  the  next  FOV. 

The  ACQUIRE  model  is  the  Army’s  standard  implementation  of  the  search  and 
target  acquisition  process  in  current  combat  simulations  (Mazz,  1998).  The  model 
determines  the  probability  of  acquiring  a  target  based  on  inputs  describing  the  target, 
sensor,  and  environment  (Mazz,  1998).  Combat  models  implement  the  ACQUIRE  model 
within  the  FOV  search  methodology  in  Figure  1.  It  is  pertinent  to  note  that  the 
ACQUIRE  model  was  originally  developed  to  model  night  vision  devices  and  not  human 
vision  (TRAC  WSMR,  2008). 
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Figure  1.  FOV  search  methodology  for  OneSAF  and  COMBATXXI  shows  the 
progression  of  decisions  which  determine  whether  a  target  detection 
occurs  and  when  to  move  to  the  next  FOV  within  the  specified  FOR.  This 
chart  is  an  abstraction  of  the  actual  search  methodology,  which  excludes 
critical  computations  required  for  each  step.  [After  U.S.  Army  Materiel 
Systems  Analysis  Activity,  2007] 

This  thesis  does  not  address  detection  performance  nor  is  it  concerned  with  a 
single  FOV  search.  This  research  disputes  the  FOR  search  in  an  urban  environment. 
Jones  and  Lai  conducted  an  FOR  search  experiment  in  which  a  4200  pixel  by  900  pixel 
FOR  was  divided  into  14  uniform  sized  600  pixel  by  450  pixel  FOVs.  They  designed  the 
experiment  to  better  understand  the  search  and  target  acquisition  process  in  an  urban 
environment  for  a  given  FOR  (Jones  &  Lai,  2007).  A  similar  division  of  the  images  in 
the  experiment  described  in  Chapter  II  of  this  thesis  would  result  in  nine  FOVs.  This 
research  will  show  that  individual  participants  do  not  approach  the  search  task  in  separate 
FOVs  using  a  windshield  wiper  pattern. 

This  study  shows  that  current  combat  simulation  models  use  search  and  target 

acquisition  methods  that  are  inconsistent  with  human  behavior.  A  large  FOR  divided  into 

uniform  FOVs  is  not  representative  of  an  individual  soldier’s  sector  of  responsibility 

(U.S.  Army,  2007).  Preset  patterns  such  as  parallel  sweeping  or  windshield  wiper 
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patterns  do  not  accurately  portray  human  behavior  and  can  drastically  affect  the  outcome 
of  a  simulated  battle.  This  research  shows  that  regardless  of  the  size  of  FOR  or  FOV, 
humans  do  not  systematically  sweep  back  and  forth  across  a  sector  of  responsibility. 
Human  participants  instead  search  for  specific  objects,  such  as  windows  and  doors,  where 
targets  are  more  likely  to  occur  (Hoffman,  2000).  Such  erroneous  outcomes  are 
detrimental  both  to  soldiers  using  the  model  for  training  and  to  commanders  who  use  the 
model  for  planning.  Furthermore,  current  models  do  not  address  how  additional 
situational  awareness  can  alter  the  search  behavior  of  individual  entities. 

3.  Current  Army  Search  Doctrine 

U.S.  Army  doctrine  does  not  address  search  in  urban  terrain  in  a  sufficient 
manner.  The  Army’s  Urban  Operations  manual  limits  the  discussion  to  the  challenges  of 
target  acquisition  in  urban  terrain.  For  example,  it  explains  how  additional  cover  and 
concealment  limit  exposure  time  of  targets  and  makes  them  more  difficult  to  identify,  but 
it  does  not  offer  a  technique  for  searching  urban  terrain  more  effectively  (U.S.  Army, 
2006). 

Chapter  8,  Urban  Areas,  of  FM  3-21.75:  The  Warrior  Ethos  and  Soldier  Combat 
Skills,  simply  discusses  movement  techniques,  entering  a  building,  and  clearing  a  room. 
Soldiers  require  each  of  these  crucial  skills  for  successful  urban  operations;  however,  this 
chapter  does  not  discuss  search  or  target  acquisition  in  urban  terrain.  Chapter  9,  entitled 
Every  Soldier  is  a  Sensor,  describes  three  daylight  visual  search  techniques.  Rapid  scan 
is  a  pattern  in  which  the  soldier  searches  100  meter  overlapping  strips  of  terrain  from  left 
to  right,  near  to  far,  until  the  entire  sector  is  covered.  Slow  scan  implements  the  same 
pattern  as  rapid  scan  with  slower  and  more  deliberate  side  to  side  movement.  If  the 
soldier  finds  no  targets  with  rapid  and  slow  scans  pictured  in  Figure  2,  he  then  uses 
detailed  search,  which  depends  on  dividing  the  sector  of  fire  into  smaller  sectors.  The 
soldier  then  incrementally  searches  small  areas  with  frequent  pauses  (see  Figure  3).  This 
search  pattern  relates  closely  to  the  search  technique  implemented  in  combat  simulation 
models.  Each  of  these  search  techniques  are  sufficient  for  rural  terrain,  but  they  do  not 
address  the  additional  complexity  in  an  urban  environment  (U.S.  Army,  2008). 
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Figure  2.  Rapid  and  Slow  Scan  Pattern.  [From  U.S.  Army,  2008] 


Figure  3.  Detailed  Search  Pattern.  [From  U.S.  Army,  2008] 

The  rifle  marksmanship  manual  (FM  3-22.9)  also  recommends  three  search 
methods.  The  soldier  employs  the  self-preservation  method  when  he  moves  into  a  new 
area.  He  quickly  scans  the  area  for  enemy  activity  and  immediate  danger  glancing  at 
specific  points  throughout  the  area.  This  method  closely  resembles  the  technique 
proposed  in  this  thesis;  however,  the  manual  does  not  specify  this  method  as  an  urban 
search  technique.  The  50-meter  overlapping  strip  method  is  identical  to  the  rapid  and 
slow  scan  patterns  discussed  previously  with  the  exception  that  the  strips  are  50  meters 
instead  of  100  meters.  Maintaining  observation  of  the  area  is  the  last  method  in  which 
the  soldier  glances  quickly  at  various  points  throughout  the  entire  area,  focusing  the  eyes 
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on  specific  points.  The  only  discussion  regarding  urban  terrain  includes  specific  firing 
positions.  The  manual  proposes  no  unique  urban  search  techniques  (U.S.  Army,  2008). 

The  Tank  (U.S.  Army,  2005),  Bradley  (U.S.  Army,  2003),  Scout  (U.S.  Army, 
2005),  and  Stryker  (U.S.  Army,  2006)  Gunnery  manuals  each  offer  the  same  two  daytime 
visual  ground  search  techniques.  The  Rapid  Scan  begins  with  the  soldier  oriented  in  the 
center  of  the  sector  and  rapidly  scanning  near  to  far.  The  soldier  then  orients  to  the  left  or 
right  and  conducts  a  similar  rapid  scan  from  near  to  far,  overlapping  the  center  sector. 
Finally,  the  soldier  orients  to  the  opposite  side  and  conducts  the  same  scan.  Slow  scan  is 
a  method  which  is  identical  to  the  50-meter  overlapping  strip  method  referenced  from  the 
marksmanship  manual  above.  Only  the  Tank  Gunnery  manual  briefly  discusses  the 
difficulties  of  target  acquisition  in  urban  terrain,  however,  it  does  not  discuss  a  search 
technique. 

The  search  techniques  referenced  above  are  sufficient  for  operations  in  rural 
terrain,  but  recent  conflicts  such  as  those  in  Somalia,  Bosnia,  Kosovo,  and  Iraq 
demonstrate  a  trend  toward  combat  in  urban  environments.  Rural  environments  consist 
of  homogeneous  natural  terrain  such  as  rolling  hills,  trees,  brush,  and  open  fields. 
Conventional  combatants  are  typically  well  camouflaged  in  rural  terrain,  and  therefore 
search  techniques  require  a  more  systematic  approach.  Such  methods  are  not  sufficient 
for  the  complex  nature  of  urban  operations.  Increased  stimuli  or  clutter  combined  with 
the  three  dimensional  aspect  of  urban  terrain  require  more  than  simple  two  dimensional 
scanning  patterns.  Search  techniques  in  urban  terrain  must  account  for  man-made  objects 
with  more  occlusions.  The  enemy  can  be  anywhere  in  a  rural  environment.  In  the  urban 
environment,  he  is  more  likely  to  present  himself  in  windows,  doorways,  behind  objects 
such  as  cars  or  the  sides  of  buildings,  or  on  rooftops.  A  preset  search  pattern  results  in 
wasted  search  effort  for  viewing  locations  at  which  a  target  is  not  likely  to  be  present 
such  as  an  empty  parking  lot  with  no  occlusions. 

The  focus  of  this  research  is  to  model  human  search  behavior,  not  an  optimal 

search  pattern.  This  thesis  proposes  recommendations  for  Army  doctrine  that  are  based 

on  professional  military  participant  fixation  data  and  do  not  account  for  detection 

performance.  Additional  research  for  improving  Army  doctrinal  urban  search  techniques 
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may  incorporate  domestic  law  enforcement  procedures.  Such  agencies  routinely  conduct 
operations  in  urban  terrain.  Although  the  scenarios  and  targets  may  differ  drastically, 
domestic  law  enforcement  urban  training  techniques  may  provide  valuable  insight. 

C.  OBJECTIVES 

Combat  simulation  models  are  beneficial  for  both  training  and  analysis,  and  as  a 
result  their  use  is  widespread.  Reliability  of  such  models  can  be  suspect  simply  because 
of  the  countless  variables  associated  with  humans  and  the  decisions  they  make.  This 
research  aims  to  augment  existing  approaches  for  the  search  process  by  incorporating  a 
model  of  human  search  behavior  and  the  role  of  additional  situational  information.  It 
also  seeks  to  lay  the  foundation  for  future  research  to  introduce  new  doctrinal  search 
techniques  in  urban  terrain. 

D.  MODELING  APPROACH 

This  research  relies  on  experimental  eye  tracking  data,  described  in  Section  II, 
from  which  fixation  determination  results.  A  fixation,  in  the  context  of  eye  movement 
analysis,  is  a  pause  on  a  portion  of  the  image  which  provides  information.  A  saccade  is 
the  rapid  movement  between  fixations  from  which  no  information  results  (Salvucci  & 
Goldberg,  2000).  Separating  data  points  into  fixations  and  saccades  is  the  first  step  to 
building  the  model. 

As  with  any  analysis  of  eye  tracking  data,  fixation  determination  is  critical.  This 
study  begins  with  an  appropriate  fixation  identification  algorithm  and  parameter 
selection.  Selecting  a  fixation  identification  algorithm  with  acceptable  parameters  is 
crucial  to  determine  where,  when,  and  at  what  the  participant  is  looking.  Various  fixation 
identification  algorithms  and  parameters  can  easily  lead  to  different  interpretations  of  the 
same  data  (Shic,  Scassellati,  &  Chawarska,  2008).  For  example,  increasing  the  velocity 
threshold  parameter  in  a  velocity-based  fixation  identification  algorithm  results  in  more 
fixations.  An  unfortunate  result  of  such  a  change  is,  for  example,  a  saccade  falsely 
labeled  as  a  fixation  point,  which  results  in  incorrect  analysis.  It  is  clear  that  this  one 
parameter  can  introduce  uncertainty  in  any  analysis  which  relies  significantly  on  fixation 
points. 
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Once  the  selected  fixation  identification  algorithm  adequately  determines  the 
fixations,  fixation  analysis  occurs  for  each  scene  across  all  participants.  Fixation 
frequency  and  order  then  determine  the  probability  of  searching  a  cell  within  a 
discretized  scene.  Mouse  click  data  determine  the  probability  of  detecting  a  target  in  a 
cell  given  a  fixation  in  that  cell.  This  data  provides  a  probability  of  detecting  each  target 
as  well  as  a  probability  of  a  false  positive  detection. 

This  thesis  applies  Myopic  Allocation  of  Search  Effort  to  model  an  individual’s 
visual  search  pattern.  The  average  fixation  size  across  all  participants  on  each  scene 
determines  the  size  of  the  discretized  regions.  The  fixations  and  detection  statistics  form 
the  basis  of  the  prior  probability  map.  The  model  uses  Bayesian  inference  to  update  the 
posterior  probabilities  after  each  small  increment  of  search  effort.  Experimental  data 
provides  the  basis  for  comparison  of  the  proposed  model  with  current  combat  models. 
This  information  is  the  basis  for  recommendations  to  improve  search  in  combat 
simulation  models  and  to  recommend  future  research  to  propose  search  techniques  for 
U.S.  Army  doctrine. 

In  addition  to  formulating  a  realistic  model  for  the  search  pattern,  this  thesis 
investigates  the  value  of  contextual  information  and  its  effect  on  search  performance. 
The  information  provided  to  participants  is  in  the  form  of  visual  situation  reports 
(SITREPS)  and  verbal  size,  activity,  location,  unit,  time,  and  equipment  (SALUTE) 
reports.  This  research  investigates  the  effect  a  SALUTE  report  has  on  detection 
probabilities  and  search  patterns. 

E.  LIMITATIONS 

There  is  significant  literature  on  the  neurological  approach  to  finding  objects  in  a 
scene.  Such  research  seeks  to  determine  how  to  find  objects  in  a  scene  based  on  the 
characteristics  of  the  scene  and  the  object.  This  thesis  uses  a  statistical  approach  to 
determine  where  and  in  what  order  participants  search  a  scene.  Determining  at  what  the 
participant  is  gazing  is  left  for  future  research. 

This  study  focuses  on  one  individual  participant  viewing  one  static  scene  at  a 
time.  At  no  time  does  a  soldier  face  an  enemy  as  an  individual  in  a  combat  situation. 
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The  modem  U.S.  Army  warrior  operates  as  a  member  of  a  team,  squad,  platoon,  or 
company.  The  field  of  view  displayed  in  the  study  is  not  representative  of  an  individual’s 
actual  sector  of  responsibility.  The  modem  warrior  is  responsible  for  a  much  more 
narrow  perspective,  so  that  he  can  pay  it  the  necessary  level  of  attention  (U.S.  Army, 
2007).  Urban  terrain  consists  of  a  vast  variety  of  stimuli  such  that  one  individual  could 
not  possibly  be  responsible  for  such  a  wide  three-dimensional  area.  He  would  instead  be 
responsible  for  a  smaller  portion  of  the  area.  Additional  study  should  focus  on  the 
interaction  of  multiple  individual  soldiers  working  in  concert. 

The  experiment  requires  participants  to  locate  hostile  targets  in  urban  terrain.  The 
instructions  to  the  participants  indicate  that  all  targets  in  the  scene  are  considered  hostile. 
A  critical  aspect  of  target  acquisition  is  target  identification.  This  study  does  not  require 
the  participants  to  identify  potential  targets  as  hostile  or  non-hostile  individuals. 

F.  CONTRIBUTIONS 

This  work  contributes  to  the  field  of  human  visual  search  in  two  main  areas.  The 
first  result  is  a  deeper  understanding  into  the  process  by  which  human  participants 
develop  a  search  pattern  based  on  top-down,  target-directed  search.  The  second  result  is 
a  new  application  of  search  theory  to  model  human  search  behavior,  which  shows  an 
improvement  on  current  techniques. 

This  thesis  shows  that,  in  the  context  of  a  target-driven  search  task,  human 
participants  do  not  search  in  a  random  pattern.  Despite  the  lack  of  a  single  distinct  search 
pattern  among  all  participants,  this  study  shows  that  participants  tend  to  repeatedly  bias 
similar  scene  locations  when  searching  for  specific  stimuli.  This  behavior  indicates  that 
the  search  process  is  a  function  of  the  environment.  Participants  search  for  objects  within 
the  urban  scene  in  which  human  targets  are  most  likely  to  exist.  Participants  expend  the 
vast  majority  of  search  effort  on  objects  such  as  windows,  doors,  behind  walls,  and  on 
roof  tops.  Furthermore,  the  participants  prioritize  this  search  effort  to  maximize  the 
likelihood  of  locating  such  targets.  This  prioritization  is  evident  through  the  examination 
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of  fixation  ordering  and  results  in  an  efficient  use  of  search  effort.  This  research  also 
shows  that  the  addition  of  contextual  information  affects  search  priority  and  drives  the 
search  behavior  in  favor  of  the  additional  information. 

The  second  important  finding  implements  the  understanding  derived  in  the 
previous  paragraph  to  develop  a  model  to  predict  human  search  behavior.  This  thesis 
reflects  a  novel  approach  for  modeling  human  search  behavior  in  naturalistic  scenes,  and 
highlights  the  shortcomings  of  current  simplistic  models.  This  work  presents  a  rigorous 
application  of  search  theory,  utilizing  probability  models  of  visual  stimuli  and  prior 
information,  and  expands  its  relevance  to  modeling  of  human  cognitive  behavior.  The 
result  is  a  statistical  model,  which  reflects  human  search  priorities  and  incorporates  the 
effect  of  added  situational  awareness. 

G.  RELATED  WORK 

Navalpakkam  and  Itti  formulate  visual  search  as  an  optimization  problem  and 
compare  the  optimal  strategy  to  human  subjects.  This  formulation  seeks  to  maximize  the 
salience  of  a  target  relative  to  distracters  using  the  signal  to  noise  ratio.  They  find  that 
humans  select  visual  cues  to  maximize  the  signal  to  noise  ratio  between  targets  and 
background  (Navalpakkam  &  Itti,  2006).  The  Myopic  Search  Model  proposed  in  this 
thesis  is  based  on  the  fact  that  humans  will  search  in  a  manner  to  maximize  the 
probability  of  detection  for  each  look.  Instead  of  focusing  on  characteristics  of  a  given 
image,  this  thesis  provides  a  higher  level  search  model  in  which  issues  such  as  image 
saliency  may  be  encompassed. 

Vogel  and  de  Freitas  implement  a  model  which  is  very  similar  to  the  Discrete 
Myopic  Search  model  proposed  in  this  thesis.  They  propose  a  model  to  optimize  the 
sequence  of  gazes  using  bottom-up  saliency  and  top-down  target  information.  The  model 
uses  a  Bayesian  sequential  decision  process  to  determine  gaze  sequence.  The  sequential 
gaze  planning  begins  with  a  prior  probability  distribution  over  all  object  locations.  The 
model  updates  this  distribution  after  each  gaze  and  the  result  is  a  policy,  or  sequence  of 
gazes,  based  on  the  most  promising  locations.  Although  top-down  target  information 
clearly  plays  a  key  role  in  participants’  search  patterns,  the  Discrete  Myopic  Search 
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model  is  statistically  based  on  participant  fixations  and  does  not  directly  consider  bottom- 
up  saliency  or  top-down  information  when  constructing  the  gaze  sequence.  Vogel  and  de 
Freitas  implement  their  model  with  state-of-the-art  object  detectors  to  locate  monitors 
and  computer  screens  within  cluttered  images.  This  is  in  contrast  to  this  thesis,  which 
focuses  on  modeling  human  behavior  rather  than  determining  the  optimal  search 
technique  to  locate  objects  (Vogel  &  de  Freitas,  2008). 

Chung  and  Burdick  propose  a  “Saccadic”  Search  strategy,  which  implements  a 
Bayesian  framework  to  update  belief  probabilities  across  a  discretized  region  after  each 
discrete  time  step.  A  cell  belief  probability  represents  the  searchers  belief  that  a  target 
resides  in  that  cell.  This  search  strategy  allocates  search  effort  to  the  cell  with  the  highest 
belief  probability  for  each  discrete  time  step,  similarly  to  the  Discrete  Myopic  Search 
model  (Chung  &  Burdick,  2007).  Despite  the  similarities  with  the  Discrete  Myopic 
Search  model  proposed  in  this  thesis,  the  search  model  presented  by  Chung  and  Burdick 
is  a  probability  model  for  search  using  a  mobile  robot  and  assumes  only  a  single  target. 
The  Discrete  Myopic  Search  model  is  a  probability  model  for  human  visual  search  of 
images  which  may  contain  multiple  targets. 

Jungkunz  proposes  a  model  for  predicting  eye  fixations  based  on  relevant  scene 
locations.  Relevant  scene  locations,  in  the  context  of  his  work,  include  such  locations 
where  human  targets  can  seek  cover  behind  walls,  in  windows,  and  in  doors  as  well  as 
locations  where  targets  can  hide  from  view.  He  develops  a  relevance  map  based  on  such 
locations  for  comparison  with  human  participant  fixation  data.  He  then  combines  the 
relevance  map  with  a  salience  map  for  improved  results  (Jungkunz,  2009).  The  relevance 
map  is  similar  to  the  prior  probability  map  constructed  in  this  thesis.  The  Discrete 
Myopic  Search  model,  however,  produces  a  fixation  pattern  based  solely  on  a  portion  of 
the  participant  data.  Although  the  Discrete  Myopic  Search  model  tends  to  fixate  on 
similar  relevant  scene  locations  as  described  above,  it  is  not  designed  specifically  to  do 
so,  nor  does  it  overtly  examine  the  contributions  of  bottom-up  and  top-down  information. 
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H.  THESIS  ORGANIZATION 


The  remainder  of  this  thesis  is  organized  as  follows: 

•  Chapter  II:  Formulation.  This  is  the  formulation  of  the  visual  search  problem 
to  include  a  description  of  the  experiment. 

•  Chapter  III:  Modeling  Allocation  of  Visual  Search  Effort.  A  thorough 
description  of  the  Discrete  Myopic  Search  model  and  its  application  to  the 
visual  search  problem. 

•  Chapter  IV:  Results  and  Analysis.  Statistical  analysis  of  the  Discrete  Myopic 
Search  model  compared  to  the  windshield  wiper  representation  of  current 
combat  simulation  models. 

•  Chapter  V :  Conclusions  and  Recommendation.  Discussion  of  the  implications 
of  experimental  results  on  the  search  process  in  current  combat  simulation 
models  as  well  as  current  Army  doctrine  and  recommendations  for  future 
research. 
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II.  FORMULATION 


This  chapter  describes  a  method  which  is  one  portion  of  a  two  tiered  Situational 
Awareness  /  Search  and  Target  Acquisition  (SA/STA)  experiment  designed  to  investigate 
SA/STA  aspects  of  current  combat  simulation  models.  Modem  soldiers  posses  vast 
amounts  of  technology  which  provides  them  a  high  level  of  situational  awareness.  The 
U.S.  Army  Training  and  Doctrine  Command  Analysis  Center  Monterey,  CA  (TRAC- 
Monterey)  developed  this  experiment  to  better  understand  the  role  of  this  additional 
situational  awareness  on  STA.  The  methodology  of  the  project  begins  by  conducting  a 
background  study  to  define  the  problem  and  refine  the  independent  variables.  The  next 
phase  is  the  conduct  of  the  experiment.  Finally,  TRAC-Monterey  will,  in  conjunction 
with  this  thesis,  analyze  the  data  to  develop  an  algorithm  for  testing  and  implementation. 
Critical  study  issues  include  determining  how  a  soldier  scans  for  targets  and  the  effect  of 
information  and  cues  on  STA. 

Tier  1  of  the  experiment  consisted  of  static  and  dynamic  stimuli  and  was 
conducted  at  the  Naval  Postgraduate  School.  Tier  2  also  consisted  of  static  and  dynamic 
stimuli;  however,  it  was  conducted  in  a  battle  lab  virtual  environment  at  Fort  Benning, 
GA.  The  ultimate  objective  is  to  develop  and  algorithm  for  implementation  into  current 
simulation  models.  Funding  for  this  project  is  provided  by  the  Soldier  Focus  Area 
Collaborative  Team  (FACT)  with  additional  funding  from  the  Office  of  the  Deputy  Chief 
of  Staff  G-3/5/7,  Modeling  and  Simulation  Directorate. 

TRAC-Monterey  is  responsible  for  the  overall  project  execution.  The  Modeling, 
Virtual  Environments,  and  Simulation  (MOVES)  Institute  at  the  Naval  Postgraduate 
School  developed  the  stimuli  for  the  16  scene  images  analyzed  in  this  thesis  and  assisted 
in  the  execution  of  the  experiment.  The  Operations  Research-Human  Systems 
Integration  Program  at  the  Naval  Postgraduate  School  assisted  in  constructing  and 
validating  the  experimental  conditions. 

This  thesis  focuses  on  the  tier  1  static  data  to  formulate  the  visual  search  problem 
as  an  optimization  model.  Each  discrete  step  of  the  model  allocates  a  small  amount  of 
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search  effort  to  a  portion  of  the  image,  which  maximizes  the  probability  of  detecting  a 
target.  The  overall  probability  of  detecting  targets  in  each  scene  increases  with  each 
discrete  look.  The  experimental  data  provide  the  inputs  to  the  model. 

The  experiment  data  from  13  participants  comprises  a  training  set  while  the 
remaining  six  participants’  data  forms  a  test  set.  The  data  from  the  training  set  builds  the 
detection  probabilities  and  fixations  for  the  model.  The  data  from  the  test  set  forms  the 
basis  for  comparison  of  the  existing  model  data  with  the  proposed  model  data. 

A.  METHOD 

1.  Participants 

Nineteen  participants  volunteered  for  the  experiment  understanding  that  no 
compensation  results  from  their  participation.  Participants  are  all  members  of  the  U.S. 
Military  and  each  had  some  level  of  search  and  target  acquisition  training.  Eleven 
participants  represent  the  U.S.  Army’s  Infantry,  Engineer,  Artillery,  Ordinance  and 
Aviation  branches.  Five  participants  represent  the  U.S.  Navy  and  three  represent  the  U.S. 
Air  Force. 

All  19  participants  are  males.  Three  participants  are  left  handed  and  the 
remaining  16  are  right  handed.  The  maximum  participant  age  is  52,  the  minimum  age  is 
26,  with  a  mean  participant  age  of  35.42  ±  5.34.  Participants  served  an  average  of  13.69 
±  5.95  years  in  the  military.  Twelve  participants  have  deployed  to  a  combat  zone,  six  of 
which  conducted  foot  patrols  and  11  of  which  conducted  vehicle  patrols.  Nine 
participants  report  having  specialized  search  training.  Thirteen  participants  report  some 
experience  with  first  person  shooter  video  games. 

2.  Equipment 

The  entire  experiment  is  conducted  on  a  Dell  XPS  720  with  an  Intel  Core  2  Quad 
2.40  GHz  processor.  The  scene  images  display  on  a  large  high  resolution  Dell  monitor. 
Participant  inputs  occur  through  both  a  standard  Dell  keyboard  and  a  standard  two  button 
mouse.  The  fixation  data  is  collected  on  a  Seeing  Machines  faceLAB4  eye  tracking 
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device  (Seeing  Machines,  2006).  The  device  records  gaze  direction  through  two  digital 
cameras  positioned  on  the  desk  in  front  of  the  participant  with  each  oriented  at  one  of  the 
participant’s  eyes. 

3.  Stimuli 

There  are  a  total  of  16  scenes  and  the  number  of  targets  in  each  scene  ranges  from 
zero  to  five.  The  experimental  design  ensures  that  scene  presentation  is  random  for  each 
participant.  Targets  may  be  completely  exposed  or  partially  occluded,  standing, 
kneeling,  or  lying  in  the  prone  position.  Eight  scenes  have  movement  (described  below), 
eight  have  SITREPS,  eight  have  SALUTE  reports,  and  eight  have  Mini-Maps.  Table  1 
displays  the  variables  for  each  scene. 


Scene  Number 

o 

o 

o 

o 

<N 

O 

o 

o 

0012 

0019 

o 

<N 

O 

o 

0021 

0024 

0027 

0033 

0035 

0040 

0041 

0042 

0043 

0044 

0045 

Total  Targets 

2 

5 

2 

3 

4 

0 

4 

0 

3 

2 

3 

1 

3 

3 

2 

3 

Movement 

0 

0 

0 

0 

0 

0 

0 

0 

1 

1 

1 

1 

1 

1 

1 

1 

Mini-Map 

1 

0 

0 

0 

1 

1 

0 

1 

1 

1 

0 

1 

1 

0 

0 

0 

SALUTE 

1 

1 

1 

0 

1 

0 

0 

0 

0 

0 

1 

1 

1 

1 

0 

0 

SITREP 

1 

1 

0 

1 

0 

1 

0 

0 

0 

1 

1 

1 

0 

0 

1 

0 

SALUTE  to 
Movement 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 

0 

Table  1.  Scene  variables  designed  to  explore  changes  in  the  search  and  target 
acquisition  process. 

Movement  in  a  scene  consists  of  one  target  appearing  and  then  disappearing  from 
the  scene.  The  movement  occurs  from  behind  objects  in  the  scene,  in  windows  or  doors, 
or  from  behind  walls.  In  each  case,  the  target  enters  the  scene  after  approximately  one 
second  remains  exposed  for  approximately  two  seconds  and  then  exits  the  scene. 

The  SITREP  is  a  static  scene  which  the  participant  views  before  he  views  the 
scene  itself.  It  provides  information  to  include  the  participant’s  location  and  orientation 
in  the  scene,  an  overhead  schematic  of  the  terrain,  and  likely  enemy  actions.  The 
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participant  may  glean  insight  into  probable  enemy  locations  based  on  the  SITREP. 
The  SITREP  scene  displays  until  the  participant  is  satisfied  and  informs  the  experimenter 
that  he  is  prepared  to  continue.  Figure  4  is  an  example  SITREP  for  scene  0000. 

The  SALUTE  report  is  a  verbal  alert,  which  refers  to  a  target  in  the  scene.  Each 
SALUTE  report  is  accurate  and  provides  information  regarding  the  location  of  a  specific 
target.  An  example  SALUTE  report  for  scene  0000  displayed  in  Figure  5  is:  “Look  at 
the  window  on  the  left.  I  think  I  see  a  helmet.”  The  SALUTE  to  Movement  row  in  Table 
1  highlights  SALUTE  reports  which  refer  to  a  moving  target.  Only  one  SALUTE  report 
refers  to  a  moving  target  in  this  experiment. 

oooo 

1.  Enter  complex  at  this  position. 

Characteristics  of  Objective: 

2.  Doorway. 

3.  Window. 

4.  Rooftop. 

Likely  enemy  action: 

Enemy  will  remain  in  hiding  unless  detected.  If 

detected,  will  return  fire  and  break  contact. 


Figure  4.  This  SITREP  for  Scene  0000  includes  the  participant’s  location,  key 

terrain,  and  an  enemy  course  of  action  (See  Appendix  A  for  Scene  0000). 

Figure  5  is  an  example  scene  with  a  mini-map.  The  mini-map  is  an  overhead 
schematic  of  the  scene  identical  to  the  schematic  from  the  SITREP.  It  provides  a  layout 
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of  the  terrain  and  the  location  and  orientation  of  the  participant  relative  to  the  scene. 
There  is  no  text  on  the  mini-map  to  highlight  key  terrain  or  an  enemy  course  of  action. 
It  always  appears  in  the  lower  left  comer  of  the  image  and  remains  for  the  duration  of 
the  scene. 


Figure  5.  Example  Scene  0000  with  mini-map  in  the  lower  left  comer.  The  small 
blue  triangle  in  the  mini-map  represents  the  participant’s  location. 

4.  Experimental  Protocol  and  Procedures 

Experimenters  greet  and  thank  each  participant  upon  arrival.  The  participant  then 
reads  and  signs  a  consent  form  and  completes  a  participant  questionnaire.  Experimenters 
then  administer  two  tests  to  ensure  proper  color  vision  and  visual  acuity.  The  first  test  is 
for  color  deficiency  using  the  Ishihara’s  Test  For  Color  Deficiency  24  Plates  Edition. 
This  test  ensures  that  participants  can  properly  discriminate  targets  from  the  background. 
The  second  test  is  to  ensure  that  the  participant  has  a  minimum  of  20/40  uncorrected 
visual  acuity.  Reflections  in  contact  lenses  or  glasses  can  obscure  the  eyes,  which  may 
produce  poor  eye  tracking  results,  so  it  is  critical  that  the  participant’s  vision  is 
uncorrected  (Seeing  Machines,  2006). 
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The  next  procedure  is  to  calibrate  the  eye  tracking  device.  This  requires  the 
physical  adjustment  of  the  equipment  to  account  for  the  various  participant  heights  as 
well  as  the  calibration  of  the  digital  cameras  and  software.  Calibration  is  necessary  for 
each  participant  because  the  eye  tracking  device  reads  the  individual’s  facial  features  and 
eye  movements. 

Experimenters  then  orient  the  participant  to  the  equipment  and  provide 
instructions  on  the  conduct  of  the  experiment.  The  instructions  include  a  brief  scenario, 
which  provides  a  background  for  the  scenes.  The  participant  has  20  seconds  to  scan  each 
scene  and  he  can  advance  the  scene  prior  to  the  20  seconds  if  he  is  confident  that  he  has 
found  all  targets.  The  instructions  inform  the  participant  that  there  are  between  zero  and 
six  targets  in  each  scene.  The  participant  indicates  a  target  detection  by  depressing  the 
left  mouse  button  on  the  stimulus  which  he  believes  to  be  a  target.  The  participant  then 
practices  using  the  experiment  equipment  on  an  example  scene.  Once  the  participant  is 
comfortable  with  the  example  scene,  the  experiment  begins. 

B.  FIXATION  DETERMINATION 

1.  Data 

Data  understanding  and  preparation  are  crucial  steps  to  performing  effective 
analysis.  The  experiment  produces  raw  data  in  the  form  of  three  separate  files  for  each 
participant.  The  eye  tracking  device  produces  the  first  file,  which  contains  numerous 
columns  of  data  for  each  frame.  Each  frame  is  approximately  16  milliseconds.  The 
pertinent  data  for  this  study  is  the  x  and  y  pixel  coordinates  of  the  intersection  of  the 
participant’s  gaze  with  the  screen.  The  second  file  contains  all  participant  inputs  and 
scene  information  with  the  associated  CPU  time.  The  third  file  correlates  each  frame  to 
the  CPU  time  to  assist  with  synchronization  of  the  first  and  second  files.  User  inputs  are 
mouse  clicks,  which  register  as  a  correct  hit  or  false  positive  detection.  The  computer 
records  the  time  and  x  and  y  coordinates  of  each  user  input.  The  scene  information 
includes  the  scene  name,  the  number  of  targets  in  each  scene,  the  time  each  scene  is 
advanced,  and  misses  when  the  participant  fails  to  click  on  a  target.  Before  fixations  are 
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determined,  information  from  each  of  these  files  must  be  fused  together  so  that  data 
points  are  associated  with  the  proper  time  and  all  information  falls  within  the  proper 
scene. 

SITREP  and  SALUTE  scenes  provide  additional  information  to  the  participant 
prior  to  the  onset  of  certain  scenes.  Blank  scenes  serve  as  a  primer  for  all  relevant 
scenes  in  order  to  prevent  improper  fixations  at  the  beginning  of  any  given  scene.  This 
information  is  important  because  the  eye  tracking  device  records  data  to  the  output  files 
for  these  scenes,  but  these  data  are  not  relevant  to  the  study. 

A  new  file  results  by  combining  the  three  original  files  for  each  participant.  This 
file  is  organized  in  ascending  order  of  machine  time  for  each  possible  event.  This  file 
retains  all  data  points,  to  include  the  irrelevant  points  between  scenes.  The  next  step  is  to 
extract  only  the  pertinent  information  from  the  file.  The  VisDataSearchExtract  code  (see 
Appendix  B)  performs  this  extraction  to  determine  fixations  and  compile  user  input  data. 

2.  Fixation  Identification  Algorithm 

There  are  several  algorithms  to  determine  fixations  and  each  has  associated 
advantages  and  disadvantages.  This  study  implements  a  velocity-based  algorithm  called 
Velocity-Threshold  Identification  (I-VT).  This  algorithm  is  particularly  desirable  for  this 
study  because  it  requires  only  one  parameter,  is  very  efficient,  and  runs  in  real  time 
(Salvucci  &  Goldberg,  2000).  The  algorithm  simply  labels  data  points  with  a  velocity 
below  the  threshold  as  fixation  points  and  data  points  above  the  threshold  as  saccade 
points.  Fixations  result  by  collapsing  consecutive  fixation  points.  Figure  6  is  a  visual 
representation  of  fixations  and  saccades  derived  from  the  I-VT  algorithm  (see  Appendix 
A  for  original  scene  image).  The  yellow  dots  represent  saccade  points,  the  green  dots 
represent  fixations  points  and  the  blue  circles  are  fixations  with  the  fixation  number 
inscribed  in  black  text.  The  targets  are  highlighted  in  red  for  the  benefit  of  the  reader. 
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Figure  6.  Visual  representation  of  participant  11  ’s  fixations  in  Scene  0002.  The 

yellow  dots  represent  saccade  points,  green  dots  represent  fixation  points, 
blue  circles  represent  fixations,  and  the  black  text  indicates  the  fixation 
number.  The  targets  are  highlighted  in  red. 


It  is  necessary  to  smooth  the  data  with  a  filter  because  velocity  based  algorithms 
are  sensitive  to  noise,  particularly  for  data  points  near  the  threshold  value  (Salvucci  & 
Goldberg,  2000;  Munn,  Stefano,  &  Pelz,  2008).  The  algorithm  uses  a  moving  average 
over  15  data  points.  In  addition,  fixations  require  a  minimum  of  five  fixation  points. 
This  also  prevents  noise  along  the  velocity  threshold  by  eliminating  single  points  which 
fluctuate  between  saccades  and  fixation  points. 


The  point  to  point  velocity  between  two  gaze  points  is  determined  by  using  the 

fix  +  h)  —  f  (x) 

forward  difference  approximation  f '( x )  =  - - - where  f(x)  is  the  pixel  location 

h 


of  the  participant’s  gaze  and  h  is  the  time  between  frames  (Blackmon,  Ho,  Matsunaga, 
Yanagida,  &  Stark,  1997).  The  measure  of  velocity  is  in  degrees  per  second;  therefore, 
some  computation  is  necessary  to  convert  the  values  from  pixels  per  frame.  In  this 

experiment,  one  degree  corresponds  to  approximately  48  pixels  and,  at  60  frames  per 
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second,  one  frame  lasts  approximately  16.67  milliseconds  in  duration.  The  resulting 
conversion  of  one  degree  per  second  yields  a  velocity  of  approximately  0.8  pixels  per 
frame.  Since  the  point  to  point  velocities  are  easily  approximated  using  the  forward 
difference,  an  appropriate  value  for  the  threshold  is  inferred  based  on  the  data  and 
exploratory  data  analysis  (Salvucci  &  Goldberg,  2000).  The  result  is  a  velocity  threshold 
value  of  0.8  pixels  per  millisecond  or  16.67  degrees  per  second.  Figure  7  compares  the 
frequency  of  various  fixation  sizes  for  velocity  threshold  values  of  0.4,  0.8  and  1.2  pixels 
per  millisecond.  The  comparison  displays  similar  distributions  for  each  velocity 
threshold  value  and  shows  that  small  changes  in  the  value  do  not  affect  the  analysis. 


Figure  7.  Comparison  of  frequency  of  fixation  durations  of  velocity  thresholds. 
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C.  DETECTION  STATISTICS 

The  detection  statistics  lay  the  foundation  for  the  prior  probability  map  and  thus 
are  essential  to  the  study.  The  probability  of  detection  for  each  target  results  by  dividing 
the  number  of  detections  for  each  target  by  the  number  of  participants.  This  information 
forms  the  likelihood  function  displayed  in  Table  2. 


y 

x 

Probability 
of  Detecting 

0  Targets 

Probability 
of  Detecting 

1  Target 

Probability  of 
Detecting  2 
Targets 

Probability 
of  Detecting 

3  Targets 

Probability  of 
Detecting  4 
Targets 

Probability 
of  Detecting 

5  Targets 

0  Targets  in  Scene 

1 

0 

0 

0 

0 

0 

1  Target  in  Scene 

0.6923 

0.3076 

0 

0 

0 

0 

2  Targets  in  Scene 

0.1538 

0.4038 

0.4423 

0 

0 

0 

3  Targets  in  Scene 

0.0897 

0.1282 

0.4358 

0.3461 

0 

0 

4  Targets  in  Scene 

0 

0 

0.0769 

0.4230 

0.5 

0 

5  Targets  in  Scene 

0 

0 

0 

0.1538 

0.7692 

0.0769 

Table  2.  Likelihood  function  p(y\x). 


The  likelihood  function  is  the  conditional  probability  of  detecting  a  specified 
number  of  targets  given  the  total  number  of  targets  in  the  scene.  For  example,  there  are 
six  scenes  with  three  targets  each.  This  means  that  there  are6*19=114  opportunities 
for  a  participant  to  detect  zero  to  three  targets  in  a  scene  with  three  targets.  The 
probability  that  zero  targets  are  detected  in  a  scene  with  three  targets  is  Pr(y=0|x=3)  = 
0.0897,  which  is  the  entry  at  (4,1)  in  Table  2. 
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III. 


MODELING  ALLOCATION  OF  VISUAL  SEARCH  EFFORT 


A.  DISCRETE  MYOPIC  SEARCH  MODEL 

Discrete  Myopic  Search  is  a  search  technique,  which  optimizes  the  probability  of 
detecting  a  target  for  each  of  k  discrete  observations.  The  given  search  area  is  discretized 
into  n  cells  in  which  a  probability  that  a  target  is  within  the  i'h  cell,  denoted  pt,  is 
assigned.  Each  successive  look  in  cell  i  has  a  probability  ql  of  not  detecting  the  target 
and  this  probability  is  assumed  independent  of  all  other  looks.  Given  kt  looks  in  the  ith 

n 

cell,  the  model  optimizes  the  probability  of  detection  p  =  ^  pt  (1  -  )  subject  to  k]  >  0 

i=\ 

n 

and  a  constraint  on  ^ kt  <  K;ki  e  (0,1,..., if)  for  n  cells  (Washburn,  2002). 

;= 1 

The  model  implements  a  greedy  algorithm  in  which  the  k  +  T'  look  is  allocated  to 
the  cell  with  the  greatest  increase  to  the  overall  probability  of  detection.  The  increase  in 
probability  of  detection  in  the  ith  cell  is  given  by: 

Pi (i  -  qki +1 )  -  Pi (i  -  qki‘ )  =  Pi (i  -  it )qk‘ 

This  implementation  produces  an  optimal  allocation  of  discrete  looks  because 

Pi(l-qkl)  is  a  concave  function  of  kt  (Washburn,  2002).  A  critical  assumption  to  ensure 

concavity  and  the  optimality  of  the  resulting  search  pattern  is  the  absence  of  false  positive 
detections.  Given  the  context  in  which  false  alarms,  such  as  engaging  a  civilian,  would 
incur  a  prohibitively  high  penalty,  the  assumption  of  no  false  positive  detection  errors  is 
considered  reasonable.  Further  research,  either  psychophysical  or  otherwise,  is  left  for 
future  study. 

Common  applications  of  a  Myopic  Search  model  include  anti-submarine  warfare 
and  Coast  Guard  search  and  rescue.  A  historic  example  is  the  search  for  the  sunken  USS 
Scorpion  in  1968.  Searchers  constructed  a  prior  probability  map  as  a  composite  of  nine 
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separate  priors.  Searchers  developed  the  nine  priors  based  on  different  scenarios 
explaining  how  and  where  the  sinking  occurred  (Wagner,  Mylander,  &  Sanders,  1999). 
This  thesis  proposes  a  novel  application  of  this  model  in  the  form  of  visual  search. 

This  study  slightly  modifies  the  model  to  accommodate  zero  to  six  targets. 
Normalization  of  the  prior  probability  map  ensures  that  the  sum  of  all  cells  equals  one.  If 
a  detection  occurs,  the  cell  or  cells  in  which  the  detected  target  resides  are  eliminated 
from  future  consideration.  This  change  allows  the  model  to  find  the  optimal  search 
pattern  with  multiple  targets. 

B  IMAGE  PREPARATION 
1.  Image  Discretization 

Discretization  of  each  scene  into  appropriately  sized  cells  is  the  first  step  in 
implementing  the  Discrete  Myopic  Search  model.  Discretization  allows  for  the  careful 
investigation  each  participant’s  fixations  and  provides  some  robustness  to  error  in  the 
eye-tracking  hardware.  The  size  of  each  cell  is  determined  by  the  average  fixation  size 
across  all  participants  for  each  scene.  This  mapping  of  fixation  size  to  cell  size  is  given 

n 

Yaf,X-  sizei 

by  /(Jot)  =  £  where  fix  =  — - for  n  fixations  in  a  scene  across  all  participants 

n 

and  l  is  the  length  of  each  side  of  the  cell.  Fixation  size  is  determined  by  the  time 

duration  of  the  fixation.  The  interpretation  of  this  implementation  of  fixation  size  is  that 
the  longer  a  participant  gazes  at  a  particular  portion  of  the  image,  the  more  stimuli  are 

present  in  that  area.  As  such,  l  is  assumed  to  increase  linearly  in  relation  to  Ju  in  order 

to  account  for  the  increase  in  stimuli  in  that  cell.  Figure  8  is  an  example  of  a  discretized 
scene.  Scene  0000  has  an  average  fixation  size  of  76  pixels,  which  results  in  375  square 
cells. 
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Figure  8.  Scene  0000  discretized. 

2.  Weighting  Fixations 

The  objective  of  this  research  is  to  develop  a  model  to  predict  an  individual’s 
search  pattern.  The  hypothesis  is  that  an  individual  will  search  in  a  manner  in  which  to 
optimize  his  ability  to  detect  targets  during  each  successive  fixation.  He  prioritizes  his 
fixation  pattern  based  on  where  he  believes  targets  will  most  likely  be  located.  This  also 
corresponds  to  proposed  schemes,  such  as  saliency-based  methods  for  visual  attention  in 
which  the  scan  pattern  is  prioritized  based  on  decreasing  saliency  (Itti  &  Koch,  2001). 
Since  participant  fixations  determine  the  prior  probability  map,  it  is  necessary  to 
prioritize  the  fixations  using  weights. 

Search  performance  is  not  constant  even  over  a  short  period  of  time  (Cooke, 
1983).  This  rate  of  change  is  not  well  defined,  but  can  be  estimated  from  the 
experimental  data.  As  a  participant  locates  targets  or  fails  to  locate  targets,  his  belief  that 
targets  remain  in  the  scene  decreases.  A  histogram  of  detections  over  time,  Figure  9, 
displays  a  pattern  of  search  performance. 
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Histogram  of  Detection  Times  [2.5  second  bins] 


Time  to  Detection  [seconds] 


Figure  9.  Search  performance  over  time  indicates  a  rapid  transition  from  initial 
fixations  when  the  scene  is  first  presented  to  initial  detections.  The 
detections  then  decrease  non-linearly  as  time  progresses. 

This  representation  of  search  performance  closely  resembles  a  Gamma 
distribution,  as  supported  by  existing  literature  regarding  detection  times  during  visual 
search  tasks  (Wolfe,  Torralba,  &  Horowitz,  2002).  This  distribution,  illustrated  in  Figure 
10,  becomes  the  basis  for  weighting  the  order  of  fixations  by  participant.  The  value  for 

1  ~x/ 

each  fixation  is  given  by  ^  ^  xa~le  //;  where  x  is  the  fixation  order  by  participant 

a  =  2  and  /?  varies  based  on  the  number  of  targets  in  the  scene.  Increasing  ft  with  the 
number  of  targets  in  the  scene  provides  a  higher  value  to  more  of  the  initial  fixations. 
This  allows  the  model  to  adjust  for  the  increase  in  visual  stimuli  and  to  account  for 
additional  fixations  on  target  locations.  The  initial  fixation  typically  occurs  towards  the 
center  of  the  scene  when  the  scene  initially  advances.  The  peak  represents  the  transition 
from  the  initial  fixation  to  the  first  detection  and  maps  to  the  peak  in  Figure  9. 
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Gamma  Transformation 
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Figure  10.  Gamma  weights  for  participant  11  ’s  fixations  in  scene  0000. 

3.  Prior  Probability  Map 

Both  the  probability  distribution  of  target  locations,  pt  values,  and  the 
probabilities  of  not  detecting  the  target  given  a  look  in  cell  i,  qt  values,  are  derived  from 

the  data.  There  is  one  p  value  and  one  q  value  for  each  cell  in  the  discretized  image 
resulting  in  two  matrices.  Probabilities  are  assigned  to  p  based  on  the  sum  of  the 
weighted  fixations  within  each  cell.  These  values  are  normalized  so  that  the  sum  of  all 
cell  values  in  a  scene  equals  one.  Probabilities  are  assigned  to  q  based  on  the  detection 
statistics.  Since  the  values  in  q  are  the  probabilities  of  not  detecting  the  target  given  a 
look  in  that  cell,  the  values  in  q  are  1-Pr(target  detection)  for  each  cell.  Figure  11 
displays  scene  0000  discretized  with  each  fixation  location  highlighted  in  yellow. 

Scene  discretization,  the  weighting  of  each  fixation,  and  the  mapping  of  each 

fixation  to  the  appropriate  cell  builds  the  p  matrix  and  the  prior  probability  map.  Figure 

12  is  a  three  dimensional  representation  of  the  prior  probability  map  for  scene  0000.  The 

height  of  each  cell  represents  the  probability  that  a  target  exists  in  that  cell.  The  color  of 

each  cell  indicates  the  height  of  that  cell  in  relation  to  other  cells.  “Hot”  red  colors 

transition  to  “cool”  blue  colors  to  indicate  the  transition  from  tall  peaks  to  short  peaks, 

which  represents  the  transition  from  higher  probabilities  to  lower  probabilities.  The 
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Discrete  Myopic  Search  model  allocates  search  effort  to  the  highest  peak  for  each 
discrete  look.  Note  how  the  concentration  of  fixations  around  the  window  on  the  left  and 
the  door  on  the  right  in  Figure  1 1  closely  resembles  the  probability  distribution  in  Figure 
12.  This  scene  has  a  SALUTE,  which  draws  the  participants’  initial  attention  to  the  left 
target.  The  larger  concentration  around  the  window  is  a  result  of  heavier  fixation  weights 
for  earlier  fixations.  The  model  shows  a  concentration  of  probability  in  the  center  of  the 
scene  as  well.  This  small  mass  of  probability  is  a  result  of  scene  transition  when  most 
participants’  gazes  are  towards  the  center.  The  value  of  the  cell  in  which  the  model 
allocates  search  effort  decreases  after  each  look.  This  occurs  because  the  more  effort  that 
is  placed  into  a  given  cell  with  no  detection,  the  lower  the  probability  that  a  target  exists 
in  that  cell. 
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Prior  Probability 


Figure  12.  Prior  probability  map  for  Scene  0000  based  on  Gamma  weighted 

fixations.  The  SALUTE  report  directs  participants’  early  fixations  to  the 
target  on  the  left.  These  early  fixations  have  greater  weights  and  results  in 
a  large  mass  of  probability  to  the  left  of  the  scene. 
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IV.  RESULTS  AND  ANALYSIS 


A.  PREDICTING  ORDER  OF  SUCCESSFUL  ACQUISITIONS 

The  focus  of  this  research  is  the  modeling  of  the  search  process  and  not  detection 
performance.  It  is  paramount  to  this  study  that  the  detection  probabilities  are  consistent 
for  each  model  so  that  detection  order  and  not  detection  performance  forms  the  basis  for 
comparison.  The  data  from  the  13  participant  training  set  provides  the  inputs 
implemented  by  the  models  below.  The  desired  outcome  is  not  to  predict  the  probability 
of  detection  for  the  test  set,  but  to  instead  predict  the  participants’  search  patterns  by 
examining  the  order  in  which  participants  locate  targets. 

1.  Measures  of  Effectiveness  (MOE) 

This  research  is  concerned  with  the  order  in  which  participants  detect  targets,  in 
contrast  to  other  studies,  which  consider  the  absolute  time  until  each  target  detection.  It 
is  difficult  to  determine  an  appropriate  MOE  with  which  to  compare  experiment  data  and 
model  data.  It  is  not  feasible  to  compare  the  actual  order  of  detection  as  there  are 
numerous  possibilities  for  each  scene.  This  study  instead  makes  the  comparison  using 
two  conditional  probabilities  of  detection  for  each  scene. 

Consider  a  scene  with  five  targets.  If  only  five  of  19  participants  detect  target 
one,  but  all  five  detections  are  the  first  detection  for  those  participants,  then  it  is 
important  to  note  that  the  probability  of  detecting  target  one  first  given  that  it  is  detected 
is  one.  If  the  remaining  14  participants  all  detect  target  two  first,  the  probability  of 
detecting  target  two  first  given  that  it  is  detected  is  also  one.  It  is  also  important  to 
consider  the  probability  of  detecting  each  of  the  targets  first  given  a  detection  in  the 
scene.  This  results  in  a  considerably  higher  probability  of  detecting  target  two  first  given 
a  detection  in  the  scene. 
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a.  MOE 1 

MOE  1  is  the  probability  of  detection  order  relative  to  all  other  targets  in 
the  scene.  It  is  the  conditional  probability  of  detecting  target  j  in  order  k  given  a 
detection  in  scene  m.  This  probability  is  given  by  the  following  formula: 

Pr(detecting  target  j  in  order  k  |  a  detection  in  scene  in)  = 


par 

y  target _detecti  .  k  m  V/, k,  m 
1=1 _ m _ 

par  target  order 

m  target _detecti  j  kmk/m 

i=l  j= 1  k= 1 

where  par  i  is  the  number  of  participants,  target  j  is  the  number  of  targets  in  the  scene,  k 
is  the  order  in  which  the  target  is  detected,  and  m  is  the  number  of  scenes. 


Participant 

Target  1 

Target  2 

1 

0 

1 

2 

0 

0 

3 

0 

1 

4 

1 

2 

5 

0 

0 

6 

0 

1 

7 

1 

0 

8 

0 

1 

9 

0 

0 

10 

0 

0 

11 

0 

0 

12 

1 

0 

13 

0 

0 

14 

0 

1 

15 

1 

0 

16 

0 

0 

17 

0 

1 

18 

0 

1 

19 

0 

1 

Table  3.  Detection  order  for  Scene  0000  from  experimental  data. 
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Table  3  displays  the  order  in  which  each  participant  detected  each  target  in 
scene  0000.  A  zero  indicates  that  the  participant  did  not  detect  the  target.  The  sum  of  all 
detections  is  13.  The  conditional  probability  of  detecting  Target  1  first  given  a  detection, 
MOE  1,  is  the  sum  of  the  ones  in  column  Target  1  divided  by  13  or  4  /  13  =  .3076.  Table 
4  shows  the  matrix  of  conditional  probabilities  for  scene  0000  where  the  (/,/)  entry  is  the 
probability  of  detecting  target  j  in  order  i  given  a  detection.  The  zero  entry  in  (2,1) 
indicates  that  Target  1  was  never  detected  after  Target  2. 


Order 

Target  1 

Target  2 

1 

0.3076 

0.6153 

2 

0 

0.0769 

Table  4.  Conditional  probabilities  of  detection  for  scene  0000  using  MOE  1 . 
b.  MOE  2 

MOE  2  is  the  absolute  probability  of  detection  order  for  each  target.  It  is 
the  conditional  probability  of  detecting  target  j  in  order  k  given  target  j  is  detected. 

Pr(detecting  target  j  in  order  k  |  target  j  is  detected)  = 

par 

y  target _detect  u j k\/j ,  k 

i= 1 _ 

par  order 

II  target detect. . .  k\/j 

;= 1  k=\ 

From  Table  3,  the  sum  of  all  detections  of  Target  1  is  four  and  the  sum  of 
detections  of  Target  1  in  Order  1  is  four.  This  results  in  a  conditional  probability  of  4  /  4 
=  1.0.  Table  5  is  the  resulting  matrix  of  conditional  probabilities  for  scene  0000. 


Order 

Target  1 

Target  2 

1 

1 

0.888889 

2 

0 

0.111111 

Table  5.  Conditional  probabilities  of  detection  for  scene  0000  using  MOE  2. 
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c.  Hypothesis  Test 

A  population  proportion  hypothesis  test  based  on  the  MOEs  above 
determines  the  extent  to  which  each  model  fits  the  experiment  data.  For  this  study,  p  is 
the  population  proportion  of  detections  of  a  specified  target  in  a  given  order  from  the 
population  of  simulated  detection  orders.  The  following  test  about  p  is  based  on  a 
random  sample  of  size  n.  Since  n  is  small  compared  to  the  population,  X,  the  number  of 
detections  based  on  the  MOEs,  has  approximately  a  binomial  distribution.  Since  n  = 
1000  is  large,  invoking  the  Central  Limit  Theorem  means  that  X  and  the  unbiased 

estimator  p  =^/n  are  both  approximately  normally  distributed  (Devore,  2008). 

The  null  hypothesis  is  that  the  model  proportion  is  equal  to  the  human 
proportion.  The  model  data  is  based  on  the  training  set  data  and  the  human  proportion  is 
determined  by  the  test  set  data.  The  test  procedure  is  constructed  as  follows: 

H0'P  =  Po 

Ha'P*Po 

P~  P  .  .  . 

z  =  ,  0  with  a  rejection  region  z  >  za/2  or  z  <  -za/2 

yJPo0--Po)/n 

where  n  is  the  sample  size,  p  is  the  population  proportion,  pa  is  the  sample  proportion, 
p  is  the  estimator  of p,  and  a  =  Pr (H0  is  rejected  when  it  is  true)  (Devore,  2008). 

The  above  test  is  valid  for  np0  >10  and  n(\  - p0)  >  10 .  Entries  for  which 
both  the  model  and  the  experiment  data  are  zero  are  considered  as  failure  to  reject  Ha . 

In  other  words,  if  target  j  is  never  detected  in  order  k  in  both  the  model  data  and  the 
experiment  data,  then  the  result  is  a  failure  to  reject  the  null  hypothesis  for  that  target. 
This  study  computes  matrices  similar  to  Tables  4  and  5  for  every  scene  using  both  the 
experiment  data  and  the  model  output.  It  then  conducts  a  hypothesis  test  on  each  entry 
comparing  the  model  probabilities  to  the  corresponding  experimental  probabilities,  which 
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results  in  a  total  of  128  hypothesis  tests  for  each  MOE.  The  proportion  of  entries  for 
which  the  null  hypothesis  is  rejected  quantifies  the  results.  This  null  hypothesis  rejection 
7,  rejections 

rate  is  given  by  -  for  each  MOE. 

2  valid _tests 

d.  MOE  Comparison 

The  two  MOEs  described  above  represent  very  different,  but  equally 
valuable  probabilities.  MOE  1  is  a  probability  relative  to  all  detections  within  a  scene. 
This  provides  a  comparison  of  each  target  detection  against  all  other  target  detections  and 
accounts  for  multiple  targets  detected  in  the  same  order.  MOE  2  accounts  only  for 
detections  of  a  specific  target.  This  MOE  is  useful  when  evaluating  targets  individually. 
Table  6  is  a  comparison  of  the  results  from  each  MOE  for  targets  for  which  a  SALUTE 
report  indicates  a  specific  location.  Since  each  SALUTE  report  highlights  a  specific 
target,  these  targets  provide  consistency  with  which  to  compare  the  MOEs.  Figure  13 
shows  that  the  two  MOEs  lack  a  consistent  pattern  and  highlights  the  value  of 
considering  both  MOEs  when  comparing  models. 


Scene 

0000 

0002 

0012 

0020 

0040 

0041 

0042 

0043 

MOE  1 

1 

0.947368 

0.333333 

0.75 

0.44444 

1 

0 

0.736842 

MOE  2 

0.307692 

0.24 

0.162162 

0.054545 

0.195122 

1 

0 

0.28 

Table  6.  Conditional  detection  probabilities  by  MOE  of  targets  detected  first  for 
which  a  SALUTE  is  provided. 
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Figure  13.  Comparison  of  MOEs  using  conditional  detection  probabilities  of  targets 

for  which  a  SALUTE  is  provided. 

2.  Model  Comparison  to  Current  Target  Acquisition  Models 

A  windshield  wiper  model  replicates  search  in  current  combat  simulation  models. 
This  model  begins  on  the  left  of  the  scene  image  and  sweeps  left  to  right  and  then  right  to 
left.  The  model  uses  the  same  predicted  probabilities  of  detection  used  by  the  Discrete 
Myopic  Search  model  when  a  target  is  encountered.  The  script  file,  which  implements 
this  model,  also  records  the  order  of  detection  for  each  iteration. 

Both  the  Discrete  Myopic  Search  model  and  the  windshield  wiper  model  are  run 
for  1000  iterations  and  the  results  are  compared  to  the  test  set  data  at  a  =  0.10 ,  a  =  0.05 , 
and  a  =  0.01  test  levels  for  both  MOE  1  and  MOE  2  where  a  =  Pr( H0  is  rejected  when  it 

is  true).  The  proportion  of  targets  for  which  the  null  hypothesis  is  rejected  is  used  to 
quantify  the  results.  The  hypothesis  test  provides  evidence  whether  to  reject  the  null 
hypothesis  in  favor  of  the  alternate  hypothesis  or  fail  to  reject  the  null  hypothesis.  Failing 
to  reject  the  null  hypothesis  is  not  equivalent  to  accepting  the  null  hypothesis,  which  is 
why  this  thesis  uses  the  rejection  rate  to  compare  the  models  (Devore,  2008). 

Tables  7  and  8  show  the  model  null  hypothesis  rejection  rate  comparison  for 
MOE  1  and  MOE  2  respectively.  These  rejection  rates  may  seem  discouraging  at  first 
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glance,  however,  one  must  keep  in  mind  both  the  difficulty  of  modeling  human  behavior 
and  the  vast  number  of  combinations  of  target  detection  order.  The  critical  result  is  the 
nearly  20%  improvement  using  MOE  1  and  5%  improvement  using  MOE  2  of  the 
Discrete  Myopic  Search  model  over  the  windshield  wiper  model. 


Model 

a=0.10 

a=0.05 

a=0.01 

Myopic  Search 

0.701 

0.701 

0.701 

Windshield  Wiper 

0.8933 

0.8933 

0.8533 

Table  7.  Rejection  rate  comparison  MOE  1 . 


Model 

a=.10 

a=.05 

a=.01 

Myopic  Search 

0.8081 

0.798 

0.798 

Windshield  Wiper 

0.8556 

0.8556 

0.8556 

Table  8.  Rejection  rate  comparison  MOE  2. 

A  large  sample  hypothesis  test  for  two  population  proportions  provides  evidence 
whether  the  Discrete  Myopic  Search  model  rejection  rate  is  less  than  the  windshield 
wiper  model  rejection  rate.  This  hypothesis  test  is  computed  using  the  following  test 
procedure. 

Ho'-Pi-P2=° 

Ha  'Pl-Pl  <0 

~  X  +  Y  m  ~  n  ~ 

P  = - = - P  \  + - P2 

m  +  n  m  +  n  m  +  n 

p.  —  n?  . 

z  =  ,  =  with  a  rejection  region  z  <  -za 

Jpq(-+- ) 

V  m  n 

where  p{  and  p2  are  the  population  null  hypothesis  rejection  rates  of  the  Discrete 
Myopic  Search  and  the  windshield  wiper  models  respectively,  m =n=l  000=sample  sizes, 
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px  and  p2  are  the  estimators  of  px  and  p2,  p  is  the  estimator  of  the  population 
rejection  rate,  and  q=\- p  (Devore,  2008). 


a=.10 

a=.05 

a=.01 

MOE  1 

-10.6932 

-10.6932 

-8.1832 

MOE  2 

-2.8399 

-3.4035 

-3.4035 

-Za 

-1.28 

-1.645 

-2.575 

Table  9.  Test  statistics  for  model  comparison  by  MOE. 

Table  9  shows  that  the  z  statistics  result  in  a  rejection  of  the  null  hypothesis  in 
favor  of  the  alternative  for  each  a  test  level  for  both  MOEs.  Furthermore,  the  p-value  is 
negligible  for  each  MOE  1  z  statistic  and  the  largest  p-value  for  MOE  2  is  .0019.  The  p- 
value  is  the  smallest  level  of  significance  at  which  H0  would  be  rejected  and  =  <X>(z)  for 

a  lower  tailed  test  (Devore,  2008).  In  other  words,  a  p-value  <  a  means  reject  the  null 
hypothesis  at  test  level  a .  This  gives  strong  indication  that  the  rejection  rate  of  the 
Discrete  Myopic  Search  model  is  less  than  the  windshield  wiper  model  for  both  MOEs. 
In  other  words,  the  Discrete  Myopic  Search  model  fails  to  reject  the  null  hypothesis  on 
more  occasions  of  detection  order  than  the  windshield  wiper  model.  This  indicates  that 
the  Discrete  Myopic  Search  model  is  more  realistic  when  compared  to  the  human 
experimental  data  than  the  windshield  wiper  model. 

B.  VALUE  OF  INFORMATION 

It  is  clear  that  contextual  information  changes  the  search  behavior  of  the 
participant,  but  it  is  very  difficult  to  interpret  the  value  of  such  information  for  this 
experiment.  The  measure  of  effectiveness  to  determine  the  value  of  information  is  the 
probability  of  detection  from  the  participant  data.  The  probability  of  detection  is 
compared  on  targets  for  which  the  experiment  provides  a  SALUTE  with  those  targets  that 
do  not  have  a  SALUTE.  This  comparison  is  difficult  to  make  because  the  scenes  with 
SALUTE  reports  are  different  than  scenes  without  SALUTE  reports.  This  thesis 
addresses  this  limitation  by  investigating  the  effect  of  target  size  and  the  number  of 
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targets  in  a  scene  on  the  probability  of  detection.  It  does  not  address  other  less 
quantifiable  factors  such  as  the  amount  of  clutter.  Additional  limitations  on  the  detection 
probability  comparison  include  certain  confounding  factors  such  as  learning,  fatigue,  and 
a  difference  in  the  number  of  targets  in  a  SALUTE  scene  versus  a  scene  with  no 
SALUTE.  Although  these  factors  were  considered  during  the  design  of  the  experiment, 
this  research  does  not  isolate  each  factor  to  make  an  objective  assessment  on  the  affect  of 
information  on  the  probability  of  detection.  The  initial  findings  within  this  section, 
although  interesting,  are  not  conclusive  and  provide  a  foundation  for  future  research. 
APPENDIX  A  contains  all  scene  images  for  a  qualitative  comparison  of  scenes  with  and 
without  SALUTE  reports. 

1.  Effect  of  SALUTE  Report  on  Probability  of  Detection 

One  may  expect  that  the  probability  of  detection  is  greater  on  a  target  for  which  a 
SALUTE  report  exists;  however,  the  following  analysis  shows  a  different  result  from 
experimental  data.  The  probability  of  detection  for  targets  for  which  a  SALUTE  report 
provides  information  is  0.5855.  The  probability  of  detection  for  targets  for  which  there  is 
no  SALUTE  report  is  0.7616.  There  are  m  =  608  opportunities  to  detect  a  target  with  no 
SALUTE  and  n  =  152  opportunities  to  detect  a  target  with  a  SALUTE  report.  Again 
invoking  the  Central  Limit  Theorem,  a  large  sample  population  proportion  hypothesis 
test,  similar  to  the  one  described  in  the  previous  section,  provides  evidence  that  the 
probability  of  detection  of  targets  without  a  SALUTE  is  greater  than  that  of  targets  with  a 
SALUTE. 

Ho'-Pl-P2=0 
Ha'P  l  -Pl<0 

z  >  z 

with  a  rejection  region  -  “ 

where  px  is  the  proportion  with  SALUTE  and  p2  is  the  proportion  without  SALUTE 
(Devore,  2008). 
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The  test  statistic  z  =  6.0393  is  greater  than  z<t  for  a  =0.10,  0.05,  and  0.01 
indicating  that  the  null  hypothesis  is  rejected  in  favor  of  the  alternative  hypothesis.  A  p- 
value  <  .0002  further  strengthens  this  test.  Participants  appear  to  perform  better  on 
targets  with  no  SALUTE  report.  These  results  are  interesting  considering  that  each 
SALUTE  report  refers  to  a  single  specific  target  in  the  scene.  Even  more  interesting  is 
the  fact  that  142  out  of  152  or  93.42%  of  the  first  five  fixations  occur  in  the  vicinity  of 
the  target  referred  to  by  a  SALUTE.  This  result  also  leads  to  the  question  whether  the 
SALUTE  report  affects  the  overall  probability  in  a  scene. 

The  probability  of  detection  for  all  targets  in  a  scene  in  which  a  SALUTE  report 
is  given  is  0.7025.  The  probability  of  detection  for  all  targets  in  a  scene  in  which  no 
SALUTE  report  is  given  is  0.7307.  There  are  m  =  323  opportunities  to  detect  a  target  in  a 
scene  with  no  SALUTE  and  n  =  437  opportunities  to  detect  a  target  in  a  scene  with  a 
SALUTE  report.  The  large  sample  population  proportion  hypothesis  test  described  above 
is  again  the  method  to  determine  whether  these  probabilities  are  equal.  The  p-value  for  a 

two-tailed  test  =  2  [l  -  <D  (|  z  |)J  (Devore,  2008). 

Ho'-Pi-P2=° 

Ha-Pi-Pi*Q 

with  a  rejection  region  z  >  za/2  or  z  <  -za/2  (Devore,  2008). 

The  test  statistic  z  =  0.7832  falls  outside  the  rejection  region  for  a  =0.10,  0.05 
and  0.01,  which  indicates  a  failure  to  reject  the  null  hypothesis.  A  p-value  of  .4354  re¬ 
enforces  this  conclusion.  This  means  that  there  is  not  enough  evidence  to  say  that  the 
probability  of  detection  in  a  scene  with  SALUTE  is  different  than  that  in  a  scene  with  no 
SALUTE. 

The  discrepancy  in  detection  probabilities  for  targets  with  and  without  a  SALUTE 
is  difficult  to  quantify.  One  possible  explanation  could  be  target  size.  The  average  size 
in  pixels  of  a  SALUTE  target  is  189  pixels  and  the  average  with  no  SALUTE  is  149 
pixels.  This  shows  that  the  discrepancy  in  detection  probabilities  is  not  due  to  the  size  of 

the  target.  Figure  14  highlights  the  fact  that  target  size  has  no  significant  impact  on  the 
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probability  of  detection  since  there  is  no  distinguishable  pattern  with  or  without  a 
SALUTE.  One  may  expect  an  increase  in  detection  probability  as  the  target  size 
increases,  however  the  proportion  of  detections  above  0.90  is  higher  for  targets  less  than 
200  pixels  compared  to  the  same  proportion  for  targets  greater  than  200  pixels. 
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Figure  14.  Probability  of  detection  for  targets  with  and  without  SALUTE  report. 

Tables  10  and  11  show  the  average  number  of  detections  and  the  conditional 
probability  of  detection  given  the  number  of  targets  in  a  scene  with  and  without  a 
SALUTE  report.  Figure  15  shows  that  participants  perform  slightly  better  in  scenes  with 
two  and  three  targets  when  a  SALUTE  is  provided  and  significantly  worse  in  scenes  with 
four  targets. 


Number 

Targets 

Frequency 
of  Scenes 

Total 

Detections 

Average 
Detections 
per  Scene 

Probability 
of  Detection 

0 

0 

0 

0 

0 

1 

1 

6 

0.3157 

0.3333 

2 

2 

50 

1.3157 

0.6578 

3 

3 

121 

2.1228 

0.7076 

4 

1 

55 

2.8947 

0.7236 

5 

1 

75 

3.9473 

0.7894 

Table  10.  Detection  statistics  for  scenes  with  SALUTE  report. 
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Number 

Targets 

Frequency 
of  Scenes 

Total 

Detections 

Average 
Detections 
per  Scene 

Probability 

of 

Detection 

0 

2 

0 

0 

0 

1 

0 

0 

0 

0 

2 

2 

49 

1.2894 

0.6447 

3 

3 

112 

1.9649 

0.6549 

4 

1 

75 

3.9473 

0.9868 

5 

0 

0 

0 

0 

Table  1 1 .  Detection  statistics  for  scenes  without  SALUTE  report. 
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Figure  15.  Detection  probabilities  for  scenes  with  and  without  SALUTE  report. 

2.  Effect  of  SALUTE  Report  on  Detection  Order 

Figures  16  through  31  display  the  eight  scenes  for  which  a  SALUTE  report  is 
given  followed  by  a  comparison  of  the  first  detection  probability,  MOE  1,  for  each  model 
and  the  experiment  data.  The  blue  bar  is  the  experiment  data,  the  green  bar  is  the 
Discrete  Myopic  Search  model,  and  the  red  bar  is  the  windshield  wiper  model.  Targets 
are  highlighted  in  red  for  the  benefit  of  the  reader.  The  SALUTE  report  is  written  in 
white  at  the  bottom  of  the  scene.  Keep  in  mind  that  the  SALUTE  report  in  the 
experiment  is  a  verbal  report  that  occurs  prior  to  scene  exposure.  The  probability  of 
detection  from  the  experimental  data  is  in  white  writing  in  parentheses  near  each  target. 
Moving  targets  are  highlighted  with  a  white  arrow. 
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In  six  of  eight  scenes,  the  SALUTE  target  is  detected  first  given  a  detection,  MOE 
1,  with  the  highest  probability  for  the  Discrete  Myopic  search  model.  This  occurs 
because  the  model  is  based  on  weighted  fixations  and  93.42%  of  the  time  one  of  the  first 
five  fixations  occur  within  one  degree  of  the  SALUTE  target.  In  only  three  of  eight 
scenes  the  SALUTE  target  has  the  highest  MOE  1  first  detection  probability  for  the 
experiment  data.  MOE  2  indicates  that  the  SALUTE  target  is  detected  first  given  that  it 
is  detected  66.29%  on  the  time.  The  first  detection  order  for  the  experiment  data  is 
greatly  affected  by  the  low  probability  of  detection  for  the  SALUTE  targets.  The 
windshield  wiper  model  only  provides  the  highest  MOE  1  first  detection  probability  to 
the  SALUTE  target  in  two  of  eight  scenes.  It  is  important  to  note  that  all  models  have  the 
correct  first  order  detection  probability  for  scene  0041  because  there  is  only  one  target, 
although  the  probability  of  detection  is  very  low  at  0.3158. 
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1  (.2105) 


2  (.4737) 


Look  at  the  window  on  the  left,  I  think  I  see  a  helmet. 


Figure  16.  Scene  0000  with  SALUTE  and  detection  probabilities. 


Scene  0000  Detection  1 


Figure  17.  Scene  0000  first  detection  model  comparison. 
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Conditional  Probability  of  Detection 


2  (.1579) 


Watch  behind  the  ladder,  I  think  I  just  saw  some  movement  there. 


Figure  18.  Scene  0002  with  SALUTE  and  detection  probabilities. 


Scene  0002  Detection  1 


Figure  19.  Scene  0002  first  detection  model  comparison. 
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Figure  20.  Scene  0012  with  SALUTE  and  detection  probabilities. 


Scene  0012  Detection  1 
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Figure  21.  Scene  0012  first  detection  model  comparison. 
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Conditional  Probability  of  Detection 


Figure  22.  Scene  0020  with  SALUTE  and  detection  probabilities. 


Scene  0020  Detection  1 


Figure  23.  Scene  0020  first  detection  model  comparison. 
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Conditional  Probability  of  Detection 


1  (.2105) 


Look  deep.  The  tower,  the  one  that's  closest  to  us,  is  that  a 
shadow  in  that  tower? 


Figure  24.  Scene  0040  with  SALUTE  and  detection  probabilities. 


Scene  0040  Detection  1 


Figure  25.  Scene  0040  first  detection  model  comparison. 
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Figure  26.  Scene  0041  with  SALUTE  and  detection  probabilities. 


Scene  0041  Detection  1 


Figure  27.  Scene  0041  first  detection  model  comparison.  All  detections  occur  first 
because  there  is  only  one  target  in  this  scene. 
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2  (1.0) 


On  top  of  the  ladder,  check  on  top  of  that  [adder. 


Figure  28.  Scene  0042  with  SALUTE  and  detection  probabilities. 


Scene  0042  Detection  1 


Figure  29.  Scene  0042  first  detection  model  comparison. 
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Scene  0043  Detection  1 


Figure  3 1 .  Scene  0043  first  detection  model  comparison. 
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3.  Effect  of  SALUTE  Report  on  Search  Pattern 

The  previous  sections  showed  statistically  how  the  search  technique  implemented 
by  current  combat  models  is  not  representative  of  human  search.  This  section  seeks  to 
show  qualitatively  how  human  participants  neither  search  scenes  randomly  nor  do  they 
waste  search  effort  in  areas  where  targets  are  not  likely  to  exist.  It  also  shows  how  a 
SALUTE  report  affects  the  search  behavior  of  the  participants,  as  well  as  the  Discrete 
Myopic  Search  model. 

The  SALUTE  dictates  the  initial  fixations  just  as  it  drives  the  detection  order  in 
the  Discrete  Myopic  Search  model.  As  noted  in  the  previous  section,  participants  fixate 
one  of  the  first  five  fixations  in  the  vicinity  of  a  SALUTE  target  with  probability  .9342. 
These  weighted  fixations  result  in  the  selection  of  the  SALUTE  target  as  the  first  location 
to  search  by  the  Discrete  Myopic  Search  model.  This  is  also  highlighted  in  the  previous 
section  where  the  highest  first  detection  probability,  MOE  1,  occurs  at  the  SALUTE 
target  location  in  six  of  eight  scenes  for  the  Discrete  Myopic  Search  model.  Figures  32 
and  33  demonstrate  how  the  model  incorporates  the  additional  information  provided  by 
the  SALUTE  report.  Figure  32  is  a  prior  probability  map  based  only  on  the  number  of 
fixations  in  each  cell.  This  representation  shows  very  similar  probabilities  associated 
with  the  window  and  the  door  in  the  scene  (See  Appendix  A,  Scene  0000).  In  contrast, 
Figure  33  is  the  prior  probability  map  based  on  weighted  fixations.  The  earlier  fixations 
carry  more  weight,  which  reflects  the  fact  that  participants  follow  the  SALUTE  report 
information  to  the  window  first. 
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Figure  32.  Scene  0000  prior  based  on  number  of  fixations  per  cell. 


Figure  33.  Scene  0000  prior  based  on  Gamma  weighted  fixations  highlights  the 
effect  of  a  SALUTE  report  on  the  probability  distribution. 
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Figures  34  to  37  show  the  progression  of  fixations  one  through  four  for  all  19 
participants  on  a  scene  with  no  SALUTE  report.  One  must  keep  in  mind  that  the  first 
fixation  occurs  in  the  center  of  the  scene  as  the  participants  are  shown  a  blank  wall  prior 
to  the  scene.  One  may  note  that  there  is  no  single  pattern  followed  by  all  participants.  It 
is  also  important  to  note  that,  although  there  is  not  one  pattern,  the  fixations  all  occur  at 
common  relevant  points. 

In  the  scenes  in  which  no  SALUTE  is  reported  (See  Figures  34  to  37),  each 
participant  arrives  at  each  target  in  his  own  pattern.  Fixations  occur  along  edges  such  as 
the  walls,  in  the  rubble,  and  around  the  window  in  the  top  left.  Despite  no  common 
pattern,  each  participant  quickly  closes  on  one  of  the  targets  within  the  first  four 
fixations.  Participants  waste  no  search  effort  scanning  from  left  to  right  or  from  front  to 
back  of  the  scene.  This  demonstrates  the  lack  of  a  systematic  windshield  wiper  patter  and 
suggests  that  the  search  pattern  is  a  function  of  the  environment. 

Figures  38  to  41  show  a  contrast  in  the  search  pattern  caused  by  the  SALUTE 
report.  The  SALUTE  report  for  this  scene  is  “Look  at  the  window  on  the  left,  I  think  I 
see  a  helmet.”  The  SALUTE  clearly  directs  the  attention  of  the  participants  to  the  target 
in  the  window.  Three  participants  shift  attention  to  the  target  in  the  door  by  the  third 
fixation.  This  occurs  because  some  participants  visually  move  more  quickly  through  the 
scene.  It  is  pertinent  to  note  that  once  again  the  search  patterns  do  not  waste  search 
effort. 
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Figure  34.  First  fixations  no  SALUTE 


Figure  35.  Second  fixations  no  SALUTE 
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Figure  36.  Third  fixations  no  SALUTE 


Figure  37.  Fourth  fixations  no  SALUTE 
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Figure  38.  First  fixation  with  SALUTE. 


Figure  39.  Second  fixation  with  SALUTE. 
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Figure  40.  Third  fixation  with  SALUTE. 


Figure  4 1 .  Fourth  fixation  with  SALUTE. 
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V.  CONCLUSIONS  AND  RECOMMENDATIONS 


This  thesis  uses  experimental  search  data  to  develop  a  realistic,  statistically  based, 
search  model  to  mimic  human  search  behavior.  Careful  analysis  of  eye-tracking  and  user 
input  data  results  in  accurate  fixation  determination  and  detection  probabilities,  which 
form  the  basis  for  a  quantitative  application  of  search  theory.  These  data  serve  as  inputs 
to  a  visual  search  application  of  a  Myopic  Search  model  with  discrete  looks.  Each  cell  in 
a  discretized  image  contains  a  probability  that  a  target  is  present  in  that  cell  based  on 
fixation  data.  Basing  the  probabilities  on  experimental  fixation  data  carries  the 
underlying  assumption  that  a  human  participant  will  prioritize  his  gaze  order  based  on 
where  he  believes  a  target  is  most  likely  to  be  present.  The  Discrete  Myopic  Search 
model  chooses  the  cell  with  the  highest  probability  for  each  discrete  look.  This 
application  results  in  the  optimal  allocation  of  discrete  search  effort  and  closely  follows 
the  search  patterns  of  human  participants. 

The  thesis  continues  by  investigating  the  effect  of  contextual  information  on  the 
search  behavior  of  human  participants.  Contextual  information  is  in  the  form  of  a  verbal 
SALUTE,  which  provides  information  on  the  location  of  a  specific  target  in  the  scene. 
This  information  clearly  affects  search  behavior  as  one  of  the  first  five  fixations  occur 
within  one  degree  of  the  target  highlighted  by  the  SALUTE  in  93.42%  of  the 
experimental  participant  data.  Interestingly,  the  probability  of  detecting  these  targets 
tends  to  be  lower  than  that  of  targets  for  which  no  SALUTE  is  given  despite  similarities 
in  target  and  scene  characteristics. 

A.  COMBAT  MODELS 

Current  combat  simulation  models  implement  a  windshield  wiper  search  pattern 
that  is  unnaturally  systematic  and  does  not  account  for  locations  that  are  more  likely  to 
contain  targets.  This  results  in  search  behavior  that  is  inconsistent  with  human 
participant  data  and  wasted  search  effort.  Regardless  of  the  size  of  the  FOR  or  the  FOVs 
contained  within  the  FOR,  a  windshield  wiper  search  pattern  is  not  representative  of 
human  search  behavior.  This  thesis  develops  a  replication  of  this  windshield  wiper 
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search  behavior  and  compares  it  with  the  proposed  Discrete  Myopic  Search  model. 
Hypothesis  testing  reveals  a  significant  improvement  in  search  performance  of  the 
Discrete  Myopic  Search  model  over  the  windshield  wiper  model.  This  improvement  is 
quantified  by  comparing  the  order  in  which  targets  are  detected  by  each  model  to  the 
order  in  which  participants  detected  the  targets. 

A  soldier  is  never  responsible  for  searching  a  large  FOR,  rather,  he  is  responsible 
only  for  a  small  FOV  (U.S.  Army,  2007).  Combat  simulation  models  should  not  require 
individual  entities  to  search  a  large  FOR  and  the  entities  should  consider  only  locations  in 
which  a  target  may  be  located  when  searching  a  smaller  FOV.  This  additional 
consideration  can  occur  through  the  design  of  the  terrain  and  the  location  of  individual 
entities  in  relation  to  the  terrain.  The  simulation  can  assign  probabilities  to  locations 
likely  to  contain  a  target  such  as  doors,  windows,  roof  tops,  and  behind  objects  in  the 
terrain. 

Contextual  information  clearly  plays  a  key  role  in  not  only  the  search  behavior  of 
individuals,  but  also  the  probability  of  detection.  It  is  common  practice  for  soldiers  to 
share  information  on  known  or  suspected  enemy  locations.  Simulation  models  should 
have  a  means  of  passing  information  among  entities  in  close  proximity.  If  one  entity  sees 
a  target,  that  entity  should  share  that  information  with  other  entities  in  the  team  or  squad 
and  each  entity  should  act  accordingly. 

B.  URBAN  SEARCH  IN  ARMY  DOCTRINE 

U.S.  Army  doctrine  does  not  propose  search  techniques  for  urban  terrain.  This 
research  shows  that  human  participants  with  military  training  and  urban  combat 
experience  choose  specific  locations  to  search  for  enemy  personnel.  These  search 
patterns  are  not  consistent  with  the  rural  search  techniques  proposed  in  the  Army  doctrine 
described  in  Chapter  I.  This  thesis  does  not  fully  investigate  the  effectiveness  of  the 
search  patterns  of  the  participants,  however,  the  level  of  expertise  of  the  participants 
provides  a  qualitative  indication  of  effective  search  performance.  The  following 
language  is  based  on  experimental  search  patterns  and  is  recommended  for  further 
investigation  for  possible  future  implementation  in  Army  doctrine. 
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The  complex  nature  of  unnatural  urban  terrain  provides  a  three 
dimensional  search  problem,  which  requires  a  more  deliberate  search 
process.  The  curvature  of  the  human  form  is  unique  to  the  straight  lines 
found  in  the  modern  urban  environment.  The  soldier  should  always 
address  the  most  immediate  threat  first.  He  should  begin  with  a  rapid  scan 
of  the  scene  using  peripheral  vision  to  find  human  figures  and  movement. 

A  more  deliberate  search  begins  with  objects  nearest  to  the  soldier.  The 
soldier  should  focus  on  any  object  which  may  provide  cover  and  or 
concealment  for  enemy  personnel.  Carefully  search  objects  such  as,  but 
not  limited  to,  automobiles,  piles  of  rubble  or  trash,  signs,  barrels,  and 
trash  cans.  Pay  special  attention  to  edges  and  comers  of  buildings  to 
include  windows,  doors  and  roof  tops. 

C.  AREAS  OF  FUTURE  RESEARCH 

The  experiment  conducted  by  TRAC  Monterey  was  a  two-tier  experiment.  The 
first  tier  produced  the  data  for  this  research  and  included  a  three  dimensional,  first  person 
shooter  style  portion  in  which  participants  navigate  through  a  virtual  urban  environment 
in  search  of  stationary  targets.  This  portion  of  the  experiment  collected  the  same  data 
including  mouse  clicks  and  eye  tracking  data.  This  study  did  not  incorporate  any  of  these 
data  as  they  pose  different  challenges  and,  potentially,  deeper  insight  into  the  search 
problem. 

Tier  2  of  the  experiment  took  place  at  Fort  Benning,  GA  with  a  virtual 
environment  and  fully  equipped  soldiers.  The  soldiers  used  actual  weapons  to  indicate 
detections  and  wore  a  head  mounted  eye  tracking  device.  These  data  can  be  analyzed  to 
advance  the  findings  in  this  thesis. 

Future  search  and  target  acquisition  research  should  incorporate  multiple 
participants  working  together,  each  with  his  own  assigned  sector  of  responsibility. 
Soldiers  never  operate  alone.  This  dynamic  may  provide  key  insight  into  the  way 
soldiers  search  when  working  as  a  member  of  a  team  or  squad.  It  may  also  provide  a 
realistic  avenue  to  study  the  effect  of  additional  situational  awareness  as  soldiers  pass 
information  among  one  another. 
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This  thesis  implements  a  statistical  approach  to  model  human  search  behavior  and 
does  not  investigate  the  characteristics  of  fixations.  This  research  may  be  extended  by 
investigating  why  certain  objects  draw  attention.  Experimental  data  shows  that 
participants  tend  to  fixate  on  windows,  doorways,  along  roof  tops,  and  along  the  sides  of 
buildings.  Each  of  these  urban  features  incorporates  straight  lines  and  comers.  Human 
figures  consist  of  curvature  around  the  head  and  shoulders,  which  is  in  sharp  contrast  to 
the  straight  lines  found  in  an  urban  environment.  Future  research  may  investigate  a 
model  which  searches  along  edges  to  find  curvature.  Additional  research  may  consider 
the  visual  contrast  between  a  human  and  the  urban  background.  Combining  the  curvature 
and  contrast  of  an  object  may  be  sufficient  to  identify  the  object  as  a  human.  Further 
research  is  required  to  distinguish  a  hostile  target  from  a  non-combatant. 

This  research  uses  target  detection  order  as  a  means  to  compare  search  patterns 
constructed  by  two  models.  This  comparison  relies  on  detection  probabilities  to  develop 
a  corresponding  search  pattern  and  implements  hypothesis  testing  to  quantify  the  results. 
Although  fixations  form  the  foundation  of  the  Discrete  Myopic  Search  model,  which 
constructs  a  search  pattern,  this  thesis  does  not  directly  evaluate  the  fixations.  Future 
research  may  implement  a  non-parametric  assessment  of  fixation  locations.  Comparing 
fixation  locations  and  ordering  may  provide  a  deeper  insight  into  the  human  visual  search 
process. 

There  is  a  clear  distinction  between  search  in  rural  terrain  versus  search  in  urban 
terrain,  which  is  not  discussed  in  current  Army  doctrine.  The  purpose  of  this  thesis  is  to 
model  human  search  performance,  not  to  identify  the  most  effective  search  strategy. 
Additional  research  is  necessary  to  compare  search  strategies  to  determine  one  that  is 
most  effective  in  an  urban  environment.  Such  research  may  consider  first  the 
effectiveness  of  search,  based  on  detection  performance.  This  research  may  consider 
proportion  of  detections,  number  of  false  detections,  and  mean  time  to  detection  as 
measures  of  search  performance.  One  may  then  determine  the  most  effective  search 
performance  and  then  evaluate  the  search  patterns. 
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This  thesis  investigates  the  role  of  additional  situational  awareness  on  search 
behavior  and  detection  performance.  The  experimental  data  provide  a  clear  indication 
that  such  contextual  information  drives  participants’  initial  fixations.  It  also  demonstrates 
that  this  information  may  serve  as  a  distraction  as  the  detection  performance  decreases 
when  a  SALUTE  is  provided.  Although  the  results  of  this  study  are  not  conclusive 
regarding  the  role  of  information  in  the  STA  process,  it  provides  a  rich  avenue  for  future 
research. 
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APPENDIX  A  SCENE  IMAGES 


This  appendix  contains  the  unaltered  scenes  from  tier  1  of  the  SA/STA 
experiment.  There  are  a  total  of  16  scenes  and  the  number  of  targets  in  each  scene  ranges 
from  zero  to  five.  Scene  presentation  is  random  for  each  participant.  Targets  may  be 
completely  exposed  or  partially  occluded,  standing,  kneeling,  or  lying  in  the  prone 
position.  Eight  scenes  have  movement,  eight  have  SITREPS,  eight  have  SALUTE 
reports,  and  eight  have  Mini-Maps.  Chapter  II  Section  A.  3.  of  this  thesis  describes  the 
scene  variations  in  greater  detail.  These  images,  as  well  as  images  in  which  the  targets 
are  highlighted  in  red,  are  available  at  http :// faculty.nps.edu/thchung/V isualSearch/scenes/. 


Scene  0000. 
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Scene  0002. 


Scene  0012. 
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Scene  0019. 


Scene  0020. 


69 


Scene  0027. 


Scene  0033. 
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Scene  0035. 


Scene  0040. 
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Scene  0041. 


Scene  0042. 
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Scene  0043. 


Scene  0044. 
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Scene  0045. 
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APPENDIX  B  CODE  FLOW  CHART  AND  DESCRIPTION 


The  flowchart  below  illustrates  the  sequence  of  events  to  transform  the  raw 
experiment  data  into  the  results  in  Chapter  4.  The  following  page  provides  a  brief 
explanation  of  each  MATLAB  script  file.  This  software  is  a  tool  which  may  be  used  by 
others  in  future  analysis  and  may  be  found  at 
http://facultv.nps.edu/thchung/VisualSearch/scripts/. 
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VisDataSearchExtract  uses  the  pre-processed  experiment  data  to  extract 
detection  information  and  compute  fixations  for  each  participant  and  each  scene. 

Hit  Stats  takes  participant  data  input  to  compute  probabilities  of  detection  and 
conditional  probabilities  of  detection. 

CellSize  takes  participant  data  input  to  determine  the  cell  size  for  each  image 
discretization.  The  cell  size  is  computed  based  on  the  average  fixation  size  for  all 
participants  for  each  scene. 

PMat  rix  reads  in  detection  probabilities  and  cell  sizes  to  compute  a  matrix  of 
probabilities  for  use  in  the  Discrete  Myopic  Search  model. 

WindshieldWiper  runs  the  model  designed  to  replicate  combat  models.  The 
result  is  a  series  of  matrices,  one  for  each  scene,  with  the  order  in  which  each  target  is 
detected  for  each  of  1000  iterations. 

DiscreteMyopicSearch  runs  the  Discrete  Myopic  Search  Model.  The  result 
is  a  series  of  matrices,  one  for  each  scene,  with  the  order  in  which  each  target  is  detected 
for  each  of  1000  iterations. 

Hypothesis  Test  computes  the  conditional  probability  of  detecting  a  target  in 
a  certain  order  given  a  detection  for  the  experiment  data  and  each  model.  The  code  then 
compares  the  experiment  data  with  the  model  data  by  conducting  a  population  proportion 
hypothesis  test  for  each  possible  combination  of  target  detection  and  detection  order. 
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